What data science can teach you about app performance & user experience. Myth #1: Crash Data is All I Need Myth #2: Release Planning Happens Once Myth #3: User Behavior & Business Insights are separate from App Performance

1. © 2016 Apteligent • www.apteligent.com
What Data Science Can Teach You About App Performance & User Experience • March 2017
Mobile App User Experience Myths, Debunked

2. © 2016 Apteligent • www.apteligent.com
Agenda
 Myth #1: Crash Data is All I Need
 Myth #2: Release Planning Happens Once
 Myth #3: User Behavior & Business Insights are separate from
App Performance
 Finally: Where is this data coming from?
Andrew Levy
Co-Founder & Chief Strategy Officer
andrew@apteligent.com
@andrewmlevy

3. © 2016 Apteligent • www.apteligent.com
Myth #1:
Crash Data is All I Need
Intractable Number of Permutations of Issues Impacting User Experience
Even if you don’t buy into this, what can data
science teach us about the cause of crashes?

4. © 2016 Apteligent • www.apteligent.com
20% of Crashes are Correlated with a Network Issue
Network issues account for a
large percentage of crashes

5. © 2016 Apteligent • www.apteligent.com
88% of Network Calls Were Successful Before a Crash
Mobile developers are typically careful to
detect and handle a failed network call.
What seems counterintuitive is that the
majority of crashes happen after
successful network calls

6. © 2016 Apteligent • www.apteligent.com
10% of Successful Network Calls Before a Crash Returned No Data
89% of the time speeds were lower than expected72% of the time latency was lower than average
Since latency was lower than average, we would expect the speed to increase. Since the majority of
calls were successful, the lower speed must be driven by a lower than expected bytes received.
bpl (speed) = (bytes out + bytes in) / latency

7. © 2016 Apteligent • www.apteligent.com
Myth #2:
Release Planning
Happens Once
• Where are we relative to our competitors?
• Gather relevant KPIs & performance benchmarks based on geography and App Store
category
• What are currently the most popular hardware & software configurations?
• Which platforms should you support and how will your app perform in those environments?
• Plan for new device and operating system launches
• Understand global bandwidth constraints
• Choose service providers and SDKs for analytics,
performance, and monetization
Take a Data-Driven Approach to Drive Your Experience Strategy

8. © 2016 Apteligent • www.apteligent.com
Example: Launch and Android Shopping App in India
Key Performance, Device & OS Metrics. Sample pulled from Apteligent Release Planner
• 60% of Android Shopping apps have a
launch time of < 2.0 seconds
• The overall average launch time is 3.09
seconds. Android Shopping apps have an
average crash rate of 3.38%, this is 35%
lower than the global average of 5.22%
• 52% of apps in this category have less
than < 1.5% crash rate
• The most popular operating system is
Android 6.0 with 25% usage share.
• The top 3 devices in India make up 12% of
the market.
• Of the top 20 devices, the worst
performing by latency are: Samsung
Galaxy Note 3 (675 ms), Intex Cloud Q11
4G (649 ms), and the OnePlus 2 (599 ms).

9. © 2016 Apteligent • www.apteligent.com
New Device Releases 2H 2016

10. © 2016 Apteligent • www.apteligent.com
Compare Yourself Against Relevant Competition & Prioritize Dev Time
89% of the time speeds were lower than expected

11. © 2016 Apteligent • www.apteligent.com
Myth #3:
User Behavior &
Business Insights are
Separate from App
Performance
Understand Why User Behavior Changes and the
Relationship Between Performance and Business Metrics
Users drop at every step of the
customer acquisition funnel

12. © 2016 Apteligent • www.apteligent.com
Poor Performance Drives a Huge Increase in Churn
• Crashes increase churn by as much as 6x
• Users are up to 8x less likely to return the next
day after a crash
• Revenue critical App Store categories like
Shopping and Finance are very susceptible
Rolling Retention
Nth Day Retention
“Hard Churn”
Day-1 Retention:
“Did they come back the next day?”

13. © 2016 Apteligent • www.apteligent.com
Relationship Between App Loads and Crashes : Churn

14. © 2016 Apteligent • www.apteligent.com
Example mCommerce Scenario
It is much more expensive to acquire a new user than to retain an existing user,
anywhere from 5x to 25x
• Customer acquisition cost is $50 x 600 means $30,000 just to win back another 600
• Assume $300 LTV, you just lost $180,000 in future revenue
• If you had a goal to gain 300 customers, and you’ve lost 600, you’ve just tripled
your customer acquisition cost to reach that goal
Quickly isolated the issue, spent X hours of an engineers time
(and $Y dollars, much less than the numbers outlined above)
A crash just caused your app to lose 600 customers

15. © 2016 Apteligent • www.apteligent.com
About Apteligent
Data Generated from Tens of Thousands of Apps
Userflow Failure Network Services Error Crashes Slow apps
Apteligent Data Platform

16. © 2016 Apteligent • www.apteligent.com
About Apteligent
Data Generated from Tens of Thousands of Apps
Transaction Failure Network Services Error Crashes Slow apps
Apteligent Data Platform
Optimize Business Result
Get Ecosystem Insight
Manage App Lifecycle
• Retention/Churn
• Detect Issues
• Determine root
cause
• Assess impact
• Prioritize
• Manage releases
• Monitor vital signs
• Benchmarks
and trends
• Carrier QoS
• Device adoption
• Bespoke analysis
• Engagement
• Monetization
• Data ingestion
• Correlation
• Prediction
• Recommendation

17. © 2016 Apteligent • www.apteligent.com
Feel free to contact me:
andrew@apteligent.com