Sept. 18, 2012 webinar on Mobile Performance Testing. Mark Tomlinson (former LoadRunner PM) and Dan Bartow (SOASTA VP Product Management) provide overview of planning and executing mobile testing that measures for Fast (front-end) and Strong (back-end) mobile systems. Recording of event will be available here: http://www.soasta.com/knowledge-center/webinars/

1. Presents:
Webinar 1

2. Expert Perspectives on Mobile Performance Testing
TODAY’S SOASTA PRESENTERS
Mark Tomlinson: West Evergreen Consulting, former LoadRunner PM
Dan Bartow: SOASTA VP Product Management
Moderator: Brad Johnson
Agenda:
• Poll question
• Embracing mobile performance testing – what's new and what's
not
• Building a performance test plan to address what really matters
• Testing and measuring for "Fast" and "Strong" all at once for mobile
and web
• Implementing a solution that does it all with agility to match the new
normal
Questions:
2

3. o Sheer Number of Devices (953M Smartphones)
o Different Operating Systems
o Scale of Global Customers (6B)
o Dynamic Content (Video, Animation, …)
o Emerging People to Machine Interfaces
Mobile performance at scale can not be
tested manually.
3

4. Mobile User Native
Experience (mUX) Apps
is affected by CPU
app & device
mUX is most Battery
performance only
affected by back Memory
end load and
network
performance
Web
browser Mobile Web Apps
App
users Web Server
Load Server
Cache App
Balanc Server
Web
er
Mobile Mobile Web Shared web &
Server
Databas
browser browser traffic mobile e
users mUX is affected by
HTTP(S) infrastructure
native app, device,
back-end and
network
performance
Fast AND Strong
Hybrid Mobile Apps
App
Real Web Server
devices Load Cache
Server
App
& large Balanc Server
Web
er
scale Native Dedicated or
Server
Databas
mobile App Web shared mobile infrastructure
e
traffic traffic
4

5. o Development teams test at a unit and component level (as always)
o Test teams load test with web-based traffic (if load testing at all)
o Ops teams monitor with services built pre-Smartphone
o No one communicates, yet agile dev is all
about communication!
5

6. 6

7. o 20 years in testing
o Performance Guy
o Small tests & Big tests
o Small & Big companies
o …speed freak.
7

8. - Old applications migrated to new mobile formats – “transformation”
- New Applications added-on to existing systems – “bolt-on”
- Entire new business units created for mobile consumer experience
- Device manufacturers are investing and expanding – alarmingly!!
- “The number of Smartphone users worldwide is predicted to exceed
1 billion by 2014” (Parks Associates, 2010)
- “Tablets to hit 100M shipments in 2012” (Source: ABI Research, 2012)
“Every customer I’ve worked with in the last 18 months
is prioritizing mobile performance testing.”
-Mark Tomlinson
8

9. WHAT’S NEW? WHAT’S NOT!
oDevices – lot’s of ‘em oThe risks of poor performance
oBattery Life & Heat oResponse Time & Volumes
oGeographic Diversity oRoot-cause Analysis
oLatencies – insanely high oCapacity Estimation
oVariable Processor Frequency oTest Planning
oBugs…lot’s of ‘em
9

10. A real story about a customer situation:
1.1,000 requests per second into the services infrastructure
2.Systems were fine-tuned, low-latency (< 250 ms)
3.Memory (in .NET CLR) was also optimized with perfection
4.They introduced a new mobile experience to the end-user
5.10% of the company started using the mobile app
6.What happened next…???
YOU MAKE THE CALL…
10

11. Enhance existing plans for mobile:
• Specific response-time goals for mobile
• Connected/disconnected test scenarios
• Mobile test lab setup & configuration
• Matrix of devices to be tested/supported
• Define end-user location and conditions
• Monitors for device’s physical resources
• Monitor client-side performance
Let’s review an example test strategy…
11

12. Ensure that new mobile plans include:
• Mobile Performance Objectives (time)
• Defined scope for device types
• Detailed explanation of end-user behavior
• Inter-app test conditions
• Carrier-specific test conditions
• Failure and recovery conditions
• Back-end system dependency and impacts
Let’s review a new mobile test plan…
12

13. Dan Bartow
13

14. Measure what
Matters
Global
Web
browser
users
Global
Hybrid Mobile Apps
Mobile Any App
Server
browser mix
Web
Load Server
users Cache App
of Balanc
Web Server
er
traffi Native Dedicated or
Server
Databas
Real c at App Web shared mobile infrastructure
e
devices any traffic
& large
scal
scale
mobile e o Dev teams test early and continuously for FREE with
traffic CTLite
o Test teams respond quickly with complete mobile
coverage
o Ops team validate and tune production infrastructure
14

15. Thanks & Q&A
Next Webinar: Sept. 27, 10 a.m. PST
“Automated Testing & Continuous
Deployment
for Mobile Apps in the Cloud”
(SOASTA & CloudBees)
Register at www.soasta.com/knowledge-center/webinars
White Papers, Webinar Recordings, Case Studies
www.soasta.com - Knowledge Center
Contact SOASTA: Contact Mark
www.soasta.com/cloudtest/ mtomlins@westevergreen.com
info@soasta.com @mtomlins
866.344.8766
Follow us: Contact Dan
twitter.com/cloudtest dbartow@soasta.com
@PerfDan
facebook.com/cloudtest
15