SlideShare uma empresa Scribd logo
1 de 35
Baixar para ler offline
MAD-LAB
Mobile Automation Devices LAB
Anand Bagmar
Software Quality Evangelist
About Me
@BagmarAnand
Blog – essenceoftesting
about.me/anand.bagmar
What do you expect
from this session?
Context
Ecosystem
Platforms
• Android (> v4.4.2)
• iPhones (> v5)
• Desktop / WAP
Regions
• Indonesia
• Malaysia
• India
• Middle-East
• ...
Dynamic Content
Search
Download
Offers
ADs
What this means ... Reality!
• Unique installs (Android) - > 1.75M
• Unique # of devices (Android) - > 13K
What this means ... Product!
• One product, One experience
• Unique / Regional / Local content
• User detection & offers based on origin
• Region
• Carrier network
• ...
What this means ... Testing!
• Video Quality, Dynamic Content, Download Expiry
• Simulating Carriers / Networks
• Consumer focus!
• Experience
• Device / OS combinations
Business Requirements
• Provide visibility into -
• What is Tested
• Overall Quality of Product
APIs
Payment
partners
CMS
....
....
Architecture
APIs
Payment
partners
CMS
....
....
Where are these?
How would you
approach
Automation for such
a product?
What Metrics make
sense for
determining Quality?
Automation
Strategy
Automation Criteria
• E2E tests should focus on Business Rules & User Scenarios
• Reports should reflect the above
• Understood by all Business & Technical stakeholders
• Generated automatically
• Should run frequently
• Every new change in the product-under-test
• On a variety of device / OS combinations
What to Automate?
Validate what was working, continues to work!
• Identify user journeys / scenarios
• Prioritize it!
Run Automation against...
Usage patterns from Analytics / PlayStore / App Store
• Identify devices that matter
• Identify devices that are “automation-friendly”
Automation Tech Stack
• Cucumber-JVM à For specifying Business Rules
• Appium à To allow integration with Android / iOS
• Gradle / Groovy à As build tool scripting language
• Jenkins à Continuous Integration Server
• TTA - Test Trend Analyzer à Trend & Failure Analysis
MVP / POC
Scale
&
Repeat
Experiment, Learn &
Evolve
Experiments
• Run tests on Emulators
• Use Cloud-based services (Local Vs Remote)
• SauceLabs
• TestObject
• Amazon Device Farm
• pCloudy
• Devices
Some Experiments didn’t work
• Emulators
• Video Player
• Location
• Cloud-based services
• Location
• Tech Stack
• Devices
MAD-LAB
MAD-LAB – Mobile Automation Devices LAB
Innovations in MAD-LAB
• Device management (selection, cleanup, app install and
uninstall)
• Parallel test execution (at Cucumber scenario level) -
maximising device utilisation)
• Appium server management
• Adb utilities (periodic ADB server disconnects)
• Custom reporting using cucumber-reports
• Video recording of each scenario and embedding in the
custom reports
• Analytics Automation
Conclusion
Learnings (so far)
Blue-print is important!
• Stepping-stone approach
Experiment!
• Know your objective(s)
• Fail-fast!
Make the Product Testable!!
References
• Sample code
• https://github.com/anandbagmar/cucumber-jvm-appium-infra
• Blog posts
• https://essenceoftesting.blogspot.com/search/label/madlab
• TTA (Test Trend Analyzer)
• https://essenceoftesting.blogspot.com/search/label/tta
Thank you!
@BagmarAnand
Blog – essenceoftesting
about.me/anand.bagmar

Mais conteúdo relacionado

Mais de Anand Bagmar

Mais de Anand Bagmar (20)

Design Patterns in Automation
Design Patterns in AutomationDesign Patterns in Automation
Design Patterns in Automation
 
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective KeynoteChange Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
Change Tyres In A Moving Car - Make Functional Test Automation Effective Keynote
 
Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)Rewrite vs Refactor (AgileIndia 2021)
Rewrite vs Refactor (AgileIndia 2021)
 
Next Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AINext Generation Functional & Visual Testing powered by AI
Next Generation Functional & Visual Testing powered by AI
 
The Best Test Automation Framework is...
The Best Test Automation Framework is...The Best Test Automation Framework is...
The Best Test Automation Framework is...
 
Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021Eradicate Flaky Tests - AppiumConf 2021
Eradicate Flaky Tests - AppiumConf 2021
 
Getting started with Appium 2.0
Getting started with Appium 2.0Getting started with Appium 2.0
Getting started with Appium 2.0
 
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
Visual Validation - The missing tip of the automation pyramid @AgileIndia2020
 
Selenium Deep Dive
Selenium Deep DiveSelenium Deep Dive
Selenium Deep Dive
 
Does your functional automation really add value?
Does your functional automation really add value?Does your functional automation really add value?
Does your functional automation really add value?
 
Measuring Coverage From E2E Tests
Measuring Coverage From E2E TestsMeasuring Coverage From E2E Tests
Measuring Coverage From E2E Tests
 
Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020Getting started with Visual Testing using Applitools - @TPC, Feb2020
Getting started with Visual Testing using Applitools - @TPC, Feb2020
 
Visual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA SymposiumVisual validation - The missing tip of the automation pyramid @ QA Symposium
Visual validation - The missing tip of the automation pyramid @ QA Symposium
 
Collaboration - A Taboo!
Collaboration - A Taboo!Collaboration - A Taboo!
Collaboration - A Taboo!
 
Visual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation PyramidVisual Validation - The Missing Tip of the Automation Pyramid
Visual Validation - The Missing Tip of the Automation Pyramid
 
Measuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback LoopMeasuring Consumer Quality - The Missing Feedback Loop
Measuring Consumer Quality - The Missing Feedback Loop
 
What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?What is Agile Testing? How does Automation help?
What is Agile Testing? How does Automation help?
 
Sharing (less) Pain of using Protractor & WebDriver
Sharing (less) Pain of using Protractor & WebDriverSharing (less) Pain of using Protractor & WebDriver
Sharing (less) Pain of using Protractor & WebDriver
 
Sharing the pain using Protractor
Sharing the pain using ProtractorSharing the pain using Protractor
Sharing the pain using Protractor
 
Test Data - Food for your Test Automation Framework
Test Data - Food for your Test Automation FrameworkTest Data - Food for your Test Automation Framework
Test Data - Food for your Test Automation Framework
 

Último

CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
anilsa9823
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
mohitmore19
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
Health
 

Último (20)

Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 
How To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.jsHow To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.js
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptx
 
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
 
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
 
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
 
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS LiveVip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
 
Software Quality Assurance Interview Questions
Software Quality Assurance Interview QuestionsSoftware Quality Assurance Interview Questions
Software Quality Assurance Interview Questions
 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
 
A Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxA Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docx
 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with Precision
 

Build your own MAD-LAB - for Mobile Test Automation