Join analyst David Gehringer of Dimensional Research and Sauce Labs in a Webinar that covers their recent research into how QA and dev engineers choose to test across emulators and real devices. Also, we’ll show you a demo of the Sauce Labs Real Device Cloud and how you can implement best practices of testing on both emulators and real devices to optimize your time and money.

1. © Sauce Labs, Inc.
Real Devices or Emulators: When to use What
for Automated Mobile App Testing
9/16/2015
Automated mobile testing across emulators and real devices

2. © Sauce Labs, Inc.
David Gehringer – Analyst,
Dimensional Research
Jack Moxon – Product
Manager, Sauce Labs
Abhijit Pendyal – Solutions
Engineer, Sauce Labs

3. © Sauce Labs, Inc.© Sauce Labs, Inc.
Agenda
 Survey Results – Mobile Emulators vs. Real Devices
 Real Device Testing Demo
 Q/A
 Automated Testing with Real Devices on Sauce Labs

4. © Sauce Labs, Inc.© Sauce Labs, Inc.
Sauce Labs: How we Solve Today’s Testing Challenges
• Scalable, secure cloud platform for fast and
affordable automated testing
• Pristine VMs: always READY
• Mobile emulators, simulators and real
devices
• 600+ platform and OS combinations, video
playback, build based UI

5. Sponsored by:5
Sponsored by:
Mobile Device Testing…
How Development and QA Really Do it.
A Global Survey
September 2015

6. Sponsored by:66
Research Goal The primary research goal was to capture hard data on current
practices of testing mobile applications. In addition, researchers
sought to understand which mobile devices were selected for the
testing pool.
Goals and Methodology
Methodology Mobile application executives, developers, and QA professionals
worldwide were invited to participate in a survey on the test processes
and how they validate mobile app performance.
The survey was administered electronically and participants were
offered a copy of this final report as a token compensation for their
participation.
Participants A total of 153 participants that develop or test mobile apps completed
the global survey.

7. Sponsored by:7
KEY RESULTS

8. Sponsored by:8
Who builds and
manages the
automated testing?
94% of Mobile App Testing Done In-House
In-house team
94%
Service provider
6%

9. Sponsored by:9
In your experience is
more testing done on
physical devices than
emulators?
72% State Most Testing is Done on Physical Devices
Yes
72%
No
28%

10. Sponsored by:10
Why are real devices
used in testing?
Select all that apply.
Real Devices Used to Verify User Experience
16%
38%
38%
49%
69%
78%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Other
Determine device usage metrics (CPU, Memory,
Storage, etc..)
Easier to test than using an emulator
Emulator is not accurate enough for my apps
Visually verify the UI
Understand true application performance

11. Sponsored by:11
By what process are
the most popular
devices determined?
Devices Used for Testing Determined by Market Share and
Application Patronage
Largest quantity of
devices that access
our applications
and websites
53%
Device market
share
40%
Number of devices
sold
4%
Other
3%

12. Sponsored by:12
How does your
company test mobile
apps?
Select all that apply.
92% of Companies Test Mobile Applications Manually
38%
50%
92%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Automated and part of continuous
integration (CI)
Automated
Manual

13. Sponsored by:13
Why is test
automation not used
by your company?
Select all that apply.
Time to Learn Automation Keeps Testers From Time
Savings Automation Provides
9%
13%
28%
30%
33%
41%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
I don’t know
Automated solutions don’t support our mobile
applications
Too difficult to build and maintain
No budget for automation
Automated solutions don’t provide enough test
coverage to be worth the effort
No time to learn how to use an automated
solution

14. Sponsored by:14
Why is a manual
testing approach still
used as part of your
process?
Select all that apply.
74% State Manual Testing is Easier
14%
2%
30%
35%
56%
74%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Other
Automation is too difficult
We don't have the expertise to automate all
tests
Our process requires manual testing initially
and automation later
Our automation solution cannot automate all
of our test cases
Some tests are too difficult to automate and
easier to do manually

15. © Sauce Labs, Inc.
The Sauce Labs Real Device Cloud
Extending mobile testing with real devices

16. © Sauce Labs, Inc.© Sauce Labs, Inc.
Sauce Labs RDC: The First Complete Mobile Solution
• Real devices and emulators
• DevOps integrated and optimized
• 100s of each device type enables:
• Instant availability
• High concurrency for speed
• Mobile web, hybrid and native apps

17. © Sauce Labs, Inc.
Real Device Cloud– Details
Instant Availability
Get access to devices instantaneously – no
waiting in long queues for devices to
become available
Massive Concurrency
Running tests in parallel can reduce your
total test time by as much as 100x
Custom CI Integrations
Integrate with leading CI tools like Jenkins,
Travis, Circle, Bamboo, and TeamCity via
custom plugins
Web, Native & Hybrid App Testing
Use Appium and Selenium to test mobile
web and native apps
Enterprise Features
Test pre-production apps, APIs and backends
with our secure tunnel and provision accounts
with Team Management and SSO
Videos and Screenshots
Instantly see videos and screenshots of
your mobile tests for Android and
screenshots for iOS.

18. © Sauce Labs, Inc.
Real Device Cloud– Details
• Announcement September 8th
• Public beta begins: September 30, 2015
• Real Device initial pricing $449 / device / month
• Sign up for real device Public Beta
http://info.saucelabs.com/rdcbetasignup

19. © Sauce Labs, Inc.
Real Device Demo

20. © Sauce Labs, Inc.
Questions?

21. Sponsored by:2121
For more information…
About Dimensional Research
Dimensional Research provides practical marketing research to help technology companies make smarter
business decisions. Our researchers are experts in technology and understand how corporate IT organizations
operate. Our qualitative research services deliver a clear understanding of customer and market dynamics.
For more information, visit www.dimensionalresearch.com
About Sauce Labs
Sauce Labs makes testing awesome. Our cloud-based platform helps developers test native & hybrid mobile and
web applications across 650+ browser / OS platforms, including iOS, Android & Mac OS X. Sauce works locally or
behind the firewall with a secure tunnel, and supports any programming language, framework or CI system.
For more information, visit www.saucelabs.com

22. © Sauce Labs, Inc.
More Info
Resources
• Sauce Labs Automated Mobile Testing Platform Data Sheet
http://bit.ly/1KgedRA
• Sauce Blog
http://tinyurl.com/owthm8d
• Sign up for real device Public Beta
http://info.saucelabs.com/rdcbetasignup

23. © Sauce Labs, Inc.
Thank you!