The document discusses the challenges of mobile performance testing. It notes that mobile introduces diverse channels and new risks compared to traditional websites. Performance impacts key metrics like conversion rates and can significantly affect business profits if not addressed. Testing mobile apps is more complex as performance can vary significantly based on network conditions, device type and location. Effective testing requires simulating real-world networks and remote users to identify performance issues before full deployment.
2. Mobile is a Very Diverse Channel
Automotive
Banking
Insurance
Professional Services
www.shunra.com
2
Healthcare
Engineering Services
Network Virtualization for Software Testing
6. Business Impact - Good performance vs. Bad
Poor performance impacts Customer adoption
vs
ING NL is a high performing mobile app that keeps increasing in
monthly installs, while BNP “Mes Comptes” which is a low
performing mobile app, is decreasing in popularity .
www.shunra.com
6
Network Virtualization for Software Testing
7. There isn’t a Single Performance Number
38.1
40
33.3
35
30
28.6
27.5
25
19.9
22.7
21.2
20
15
20.5
20.2
WiFi
17.5
17.2
3G
13.9
3G Busy
8.8
10
6.2
5.3
5.3
5
4.6
0
Launch App
Auto
Auto
Accident
Dental
Home
Get Life
Insurance
Quote
Transaction Response Times of an Insurance Mobile App [Sec]
www.shunra.com
7
Network Virtualization for Software Testing
8. What Happens under Mobile Load?
www.shunra.com
8
Network Virtualization for Software Testing
9. MyAT&T Mobile App – Prod Vs. Test
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
HP - Virtual Users
Hi Speed Lab Network
myAT&T – Dev/Test
www.shunra.com
10. Impact of Remote Users
Local Load Test Results
VUsers
Multi Site Load Test
Results
VUsers
Avg. TRT
Avg. TRT
Why?
www.shunra.com
Network Virtualization for Software Testing
10
11. What Happens under Mobile Load?
WiFi
3G
Each User Slows Down
Server Resources get Exhausted Quicker
www.shunra.com
11
Network Virtualization for Software Testing
12. The story of a mobile app
Testing - Singapore
Development - Pune
Deployment - Spain
www.shunra.com
Network Virtualization for Software Testing
13. Different Geographies have Different
Network Conditions
www.shunra.com
Network Virtualization for Software Testing
14. When it is time to Deploy
How will it Perform?
What will the App reviews
be?
We better run a field pilot for
a month!!!
www.shunra.com
Network Virtualization for Software Testing
15. When it is time to Deploy
What Do you Mean Pilot?
Sprint = 4 weeks
Week 1
Sprint
Planning
Meeting
Week 2
Week 3
Sprint n
Sprint
Review
Sprint n – Development / QA
Sprint n + 1 - Preparation
www.shunra.com
Week 4
Sprint n
Deployment
to Production
on following
Wednesday
Sprint n –
Integr. Test
Sprint
n+1
Network Virtualization for Software Testing
Notas do Editor
Brand ImpactHigh correlation between app performance and brandImmediate feedback on the app storeVicious loop: Poor reviews -> decrease in downloadsRevenue ImpactHigh correlation between slow response and abandonment rateAlternative channel cost when users abandon a mobile appe.g. – mobile banking abandonment increases ATM costs, Branch visit costs, Call Center costsTime to marketLong field pilots are not adequate in an Agile-Rapid-Mobile app life cycle
As I mentioned, I’m going to use one of your key initiatives to demonstrate how shunra can help.I’d like to talk about your mobile app strategy, in particular the US banking mobile app….
With Shunra you know exactly what mobile conditions are at NG16 8YQ or anywhere else in the market for that matter