World Class Optimization: Benchmarking 1,000+ Companies
•
2 gostaram•2,552 visualizações
Innovation is required for any growing organization, but nearly impossible without a robust experimentation strategy. This webinar presents you with discoveries on best practices gleaned from an expansive data set spanning over 100,000 experiments in 2016. Watch this on-demand webinar to learn: - What traits define the best testing organizations - How can you drastically increase your testing performance - What are the right team goals to measure your testing program
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a World Class Optimization: Benchmarking 1,000+ Companies
Semelhante a World Class Optimization: Benchmarking 1,000+ Companies (20)
Mais de Optimizely
Mais de Optimizely (20)
Último
Último (20)
World Class Optimization: Benchmarking 1,000+ Companies
- 1. @Hazjier#ConvCon World-Class Optimization: Benchmarking 1,000+ Companies 2016 State of Global Optimization @Hazjier Hazjier Pourkhalkhali Global Lead, Optimizely Strategy
- 2. 40% 19% 10% 7% 5% 4% Rest Market share Alexa 10K Websites We are the world’s largest digital laboratory
- 4. We have researched the global state of Optimization for years Step 02 Interviews Customer Success ResearchOptimization Substantial site-wide personalization Experimentation Prototypes and custom integrations Global Optimization Benchmark Industry surveys Top Revenue Experiments Driving adoption Partner services Optimization services 500+ Business Reviews Customer feedback Expert feedback
- 5. Over 100,000 Experiments Across all verticals Spanning most regions Covering all site sizes ~30% ~20% ~15% ~10% ~5% ~20% Retail Media High Tech Travel Finance Other § 250K variations § 1000+ companies § At least 1K visitors / variation § Includes variation code NORTH AMERICA EMEA ASIA PACIFIC Alexa 1K Alexa 10KAlexa 100K Other 2016 Global Optimization Benchmark @Hazjier
- 6. 1 Drive deep insights 3 Ensure technical resources 2 Develop multiple solutions 4 Analyze thoroughly
- 7. 1 Drive deep insights
- 8. 1.00 1.32 1.48 NO ANALYTICS ANALYTICS ANALYTICS + HEATMAPS Significant upliftExperiment win rates based on Analytics instrumentation Indexed win rate @Hazjier
- 9. Know your revenue drivers Revenue: $10M Revenue Per Visitor: $2.50 Average Order Value: $100.00 Quantity: 2.00 Price: $50.00 Conversion Rate: 2.5% Add to Cart: 5.00% Cart Checkout: 50% Visitors: 4M Acquisition: 3M Retention: 33% @Hazjier
- 10. Add to Cart: 5.00% Cart Checkout: 50% Homepage Flight Results Book Flight Passenger Details Flight Options Payment Page Step 45% 30% 65% 65% 90% 70% 40% 40% 15% 20% 5% 25% 15% 30% 20% 15% 5% 5% Exit Rate Divergence Completion Flight Results
- 11. What pain points do users experience? • What do users want to do? • Where do they focus? • Is the page clear? Refine search Select flight Change date Sort flights @Hazjier • What do surveys indicate? • Do certain groups behave differently? • What else can you learn from: • Customer interviews • UX research • Customer service discussions
- 12. 2 Develop multiple solutions
- 13. There is a big problem with “A/B Testing” @Hazjier
- 14. 2 Variations 4 Variations 5 Variations >6 Variations 3 Variations A B C D E F @Hazjier
- 15. Three-fourths of all experiments only have 2 variations 2 Variations 3 Variations 4 Variations 5 Variations >6 Variations 77% 14% 5% 2% 1% @Hazjier
- 16. Testing 5 or more variations can improve your win rate by 75% +71% +75% +48% +32% 25% 33% 37% 44% 43% +75% 2 Variations 3 Variations 4 Variations 5 Variations >6 Variations Significant uplift Significant reduction Inconclusive @Hazjier
- 17. We can say with statistical certainty that more variations is better! 18,956 75,055 4,619 13,869 1,815 4,859 1,005 2,275 566 1,310 >99% winner >99% winner >99% winner >99% winner NUMBER OF VARIATIONS STATISTICALLY SIGNIFICANT UPLIFTS EXPERIMENTS EXPERIMENT WIN RATE IMPROVEMENT STATISTICAL SIGNIFICANCE STATUS 5 Variations is currently beating 2 Variations by 57% BEHAVIOR Testing More Variations The percentage of experiments that reached a statistically significant uplift +71.1% +47.9% +31.9% 25.3% 33.3% 37.4% 44.2% 43.2% +74.9% 2 Variations 3 Variations 4 Variations 5 Variations >6 Variations @Hazjier
- 18. 3 Ensure technical resources
- 19. We analyzed a quarter million variations by the number of lines of code @Hazjier
- 20. 8% 92% Over half of all variations are written with 5 or fewer lines of code 1-5 LOC 6-10 LOC 11-20 LOC 21-50 LOC 51-100 LOC 101-200 LOC >200 LOC 51% 11% 10% 10% 4% 2% 2% @Hazjier
- 21. Hire a developer! 1-5 LOC 6-10 LOC 11-20 LOC 21-50 LOC 51-100 LOC 101-200 LOC >200 LOC 17% 21% 25% 29% 35% 37% 36% +112% +106% +118% Hire a developer! Complex variations win 2.1–2.2x more often! @Hazjier
- 23. Analyze your experiments thoroughly 6 Goals 8 Goals 4 Goals 2 Goals 3 Goals 1 Goals 7 Goals 9 Goals 5 Goals >10 Goals 19% 14% 7% 3% 2% 27% 10% 5% 3% 10% @Hazjier
- 24. Analyze your experiments thoroughly 6 Goals 8 Goals 4 Goals 2 Goals 3 Goals 1 Goals 7 Goals 9 Goals 5 Goals >10 Goals 18% 23% 32% 40% 44% 15% 28% 38% 44% 44% @Hazjier
- 25. Track your micro-conversions! Conversion Rate Product page landing Site search Sub- category Filter interaction Product View Rate Products Viewed / Session Product Relevance Size/Type Interaction Add-to-Cart Cart Checkout Field attempt rate Time per field Field success rate Sign-up / Log-in Shipping Details Payment @Hazjier
- 26. Visitors per variation 1K-10K 10K-50K 50K-100K 100K-200K 200K-500K 500K-1M 1M+ 1K - 10K - 50K - 100K - 200K - 500K - 10K 50K 100K 200K 500K 1M Over 1M 18% 22% 26% 21% 13% 24% 25% @Hazjier
- 27. Best Practices of World-Class Optimization United States and Canada 12% 10% 8% 15% 3 88% 90% 92% 85% 97% % of Accounts % of Experiments % of Variations % of Experiments % of Variations Heatmaps + Analytics 4+ Variations >50 Lines of Code 7+ Goals 100K+ Visitors @Hazjier
- 28. 2 Traffic 1 Resources 3 Time § Hire employees § Ask internally § Contract externals § Acquire traffic § Improve retention § Make drastic changes § Choose high baseline goals § Wait longer, test concurrently
- 29. Best Practice ExperimentAverage Experiment 103,289 § 100K+ Visitors / variation § 50+ LOC / variation § 7+ Goals § 4+ Variations 26% § 1K+ Visitors / variation § 1+ LOC / variation 77% 183 Significant uplift Significant reduction Inconclusive What true “Best Practice” looks like @Hazjier
- 30. 1 Drive deep insights 3 Ensure technical resources 2 Develop multiple solutions 4 Analyze thoroughly Questions?
- 31. Next Steps Watch a Live Demo OPTIMIZELY.COM/LIVE-DEMO 21 Sign up for a Free Trial OPTIMIZELY.COM/GET-STARTED Join the Community OPTIVERSE.COM 3
- 32. Thank you! @Hazjier@Hazjier Hazjier Pourkhalkhali Global Lead, Optimizely Strategy