Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Mobile Testing and Mobile Automation at eBay
1. Mobile Testing and Mobile
Automation at eBay
Michael Palotas & Dominik Dary
Quality Engineering Europe
eBay International AG
September 2012
2. eBay Inc. Facts
Founded in 1995
Based in San Jose, California
27770 Employees worldwide
3. Mobile Facts
Mobile Facts: Mobile Fun Facts:
100,000,000 downloads Every 30 seconds, a woman’s
handbag is purchased on eBay
of eBay Inc’s apps Mobile.
Every 2 minutes, a tablet is
Mobile transaction volume bought through eBay Mobile.
will double in 2012 to 10
billion USD Every week, more than 8,000
cars and 340,000 car parts are
1,7 million items sold through eBay’s mobile
apps.
are listed with using mobile
7,493 tablets sold weekly from
apps every week April – June;
enough to cover the surface of
3 Olympic-sized pools.
4. Who we are
Michael Palotas
Head of Quality Engineering Europe
E-mail: mpalotas@ebay.com
Dominik Dary
Staff Software Engineer in Test
E-mail: ddary@ebay.com
5. Agenda
1. Challenges in Mobile Testing
2. Manual Mobile Testing
3. Our Requirements for Mobile Test
Automation
4. Test Automation of the Mobile
Web, Android Apps & iOS Apps
5. Conclusion
9. Manual Mobile Testing
Feature testing is manual
Concentrate on top 5 devices
Functionality, Layout, User
Experience and Localization
In depth EU specific domain
knowledge
10. The importance of parallel Test Execution
• Multiple Apps
• 12 European Sites
• iOS Devices
• iPhone vs. iPad
• Version
• Android Devices
• Manufactures
• Screen Sizes
• OS Modifications
Picture Source: http://www.flickr.com/photos/ippei-janine/483345027/
12. Mobile Test Automation Requirements
• Use the Load Balancer Features of the Selenium
Grid2 to execute the End-to-End Tests in parallel
• Manage multiple applications / versions / languages
• The Mobile App should not need to be modified
(e.g. like adding a test server)
• Mobile App Runtime Inspection Support
13. Testing of eBay‘s Mobile Products
Android Mobile Web Android Native App iOS Native App
14. Test Automation of the Mobile Web
We use Selenium and specifically
the AndroidDriver and the
iPhoneDriver
Test Cases are written in Java and
are similar to web tests
Tests are only executed on
Emulators
http://code.google.com/p/selenium/wiki/IPhoneDriver
http://code.google.com/p/selenium/wiki/AndroidDriver
15. Android Mobile Web Test Infrastructure
TestNG
Test
Selenium Grid http
http Jetty Server
Hub
Android
Driver
WebView
Android Server
QE Machine Backend Device
16. Page Tests
Flow Objects
Objects Mobile Web Test Case
17. Test Automation of the Android Apps
We have created the Open Source
project Calabash-Driver
Calabash-Driver integrates
Calabash-Android into the Selenium
Grid
Calabash-Driver offers a convenient
Webdriver like Java API for writing
tests http://calabash-driver.github.com
18. Test Automation of Android Apps
Selenium Grid
TestNG Hub
Test
http
http
Instrumentation
http Backend
Calabash
Calabash Android
Jetty Server
Driver Connecto
r
eBay App
ADB
Connec- USB
tion
QE Machine Device
Calabash Server
Backend
19. Dialog Tests
Flow Objects
Objects Android App Test Case
20. Test Automation of the iOS Apps
We have created the Open Source
project iOS-Driver
iOS-Driver integrates Apple’s official
tool UI Automation into the Selenium
Grid
iOS-Driver offers a convenient
Webdriver like Java API for writing
tests https://github.com/freynaud/ios-driver
21. Test Automation of iOS Apps
Selenium Grid
TestNG Hub
Test
http
http
Instru-
iOS eBay App
Jetty Server
ments
Driver Manager
Apple‘s UI
Automa-
tion
QE Machine iOS Simulator
iOS-Server
Backend
22. Dialog Flow Tests
Objects Objects iOS App Test Case
24. Advantages of Calabash-Driver & iOS-Driver
Solutions are integrated into the Selenium
Grid which enables us to scale
The concept of automated testing the Web,
Win32, mobile Web and native apps is the
same
Fast Rampup for the testers and quick
return on investment
Both Solutions are supporting the
interaction with elements based on the
L10n resource bundle key
25. Fork us at Github
http://calabash-driver.github.com
https://github.com/freynaud/ios-driver
26. Conclusion
No single automation solution for mobile
Web, Android and iOS apps – and that’s ok!
Invest into Scalability by Virtualization
Buy the top devices
Open source is your best friend!
Interested in our solutions? Contribute to our projects!
Call out shapes can be found on pg. 56 and 57Arrows can be found on pg. 59
Talk hereaboutwhywefocus on parallel execution.Weare asmallteamWehavetotest multiple appsfor multiple sites on different platformsWewantedtoadaptthelearningsfromthe web to mobile‚Build in Scalability“ veryimportant
Talk hereaboutwhywefocus on parallel execution.Weare asmallteamWehavetotest multiple appsfor multiple sites on different platformsWewantedtoadaptthelearningsfromthe web to mobile‚Build in Scalability“ veryimportant
Image size: 9” (width) x 5.17” (height)Right click and select “Change picture” to replace image
Here Iwouldliketoshowsomeoftheautomationvideosofthecheckoutflowor do a live demo.