Creating an effective A/B testing strategy can be a real challenge. We joined the App Radar Webinar to answer all the hot-button questions:
How to craft your A/B strategy from scratch
How to design hypothesis and evaluate results
How to get real techniques and strategies to follow
You’ll find all the answers you need provided by Natalie Ostapovich, Customer Support & Account Manager at SplitMetrics, and Elisa Mourão, Marketing Manager at App Radar in the short and easy-learn slides and infographics.
3. Agenda
Pre-Launch A/B Test
A/B Testing in App Stores
01
02
splitmetrics.com appradar.com
Timeline
Hypothesis
What to test
How to run A/B tests
How to Analyze Results
6. Why and How to A/B test in Pre-Launch Phase?
Validate your app idea;
Find positioning;
Find target audience;
Prioritize app features;
Polish store listing.
splitmetrics.com appradar.com
7. How to interpret results and move on?
Conversion Rate;
Performance in different
audiences;
Bounce Rate;
Explored sections from
the store listing.
splitmetrics.com appradar.com
9. How to come up with a strong hypothesis?
Research Phase
App Product Page vs. Category and Search Page
What type of experiment should be aimed for clear results ?
How many hypothesis should be tested at the same time?
splitmetrics.com appradar.com
10. What elements to test?
Icon, screenshots, video
How about the other elements in the store listing?
splitmetrics.com appradar.com
11. Testing the App Icon
splitmetrics.com appradar.com
22%
Uplift
Color palette
12. Testing the App Screenshots
splitmetrics.com appradar.com
24%
Uplift
Portrait vs. Landscape
Colors
Order
Features vs. Gameplay
13. Testing the App Video
splitmetrics.com appradar.com
17%
Uplift
Does an app video increase
conversion rates?
Which videos show better
performance?
How many videos should I
have in my store listing?
14. How to Run A/B Tests
splitmetrics.com appradar.com
For how long should I run my test?
How does targeting work with A/B testing?
What is enough traffic for an experiment?
15. splitmetrics.com appradar.com
“It normally takes at least 50 ‘Clicks to Install’ per
variation to get statistically significant results. On
average, you need 400-500 visitors per variation
for a test on your app’s product page and
800-1000 users per variation if you run an
experiment on the search or category page.
SplitMetrics
”
16. Analyzing Results
What is and how to interpret the Confidence Level?
How to choose a winner?
Main KPIs
01
02
splitmetrics.com appradar.com
Visitors per variation
Clicks to install per variation
Time to click to install
Engagement rate in the store listing
Improvement Metric
Retention rate
03
17. Analyzing Results – Best Practices
What if my new variation loses?
What if I don't have an answer after the test?
Should we use the same winning creatives to the App Store
and Google Play or should we a/b test per store?
splitmetrics.com appradar.com
18. Thank You!
Questions?
Natalie Ostapovich
Customer Support & Account
Manager @ SplitMetrics
Elisa Mourao
Marketing Manager
@ App Radar
Sign up: appradar.com
Follow us on social media:
appradar
appradar
appradar_com
App_rad ar
App Radar
splitmetrics.com
searchadshq.com
splitmetrics.com/agency/