O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

How to Successfully Run Your First Website A/B Test

5.736 visualizações

Publicada em

What is A/B testing? A SCIENTIFIC METHOD OF PROOF
WHAT IS A/B TESTING FOR MARKETING
Where do you start testing? THE DON’TS
WHERE DO YOU START TESTING?
DON’T JUST GUESS
SUB OPTIMIZE • Button colors • Small copy changes • Small layout changes • Fonts, font size
WATCH OUT FOR BUZZ • Don’t just test what the gurus say • What works for others may not work for you
PREMATURE TESTING • Testing without traffic • Before you know what and why you are testing
Where do you start testing? THE DOS
DO YOUR HOMEWORK You have to put in the work to get great results • Learn as much as you can about A/B testing • Study the great people out there • What’s their process and insights?
A framework for success The basis of a successful testing program
Research Know what to test and see the greatest improvements
DIVE INTO YOUR DATA
FIND YOUR BIGGEST OPPORTUNITIES
INVESTIGATE THE OPPORTUNITY TO UNDERSTAND IT
FORM A HYPOTHESIS By doing A, B will happen, Because of C
BUILD THE TEST
LAUNCH THE TEST VS. Control Variant
THE TEST VS. +24.01% increase in registrations!! 100% Significance
DOCUMENT AND SHARE YOUR FINDINGS Documenting all your tests and results • Increases company transparency • Helps others know where tests are going on • Makes it easier to call tests • Makes learnings available for everyone in company
REPEAT THE PROCESS AS OFTEN AS POSSIBLE
Control vs. Variant
“Only 1 out of 8 A/B tests have driven significant change.” Noah Kagan AppSumo*
“The goal of a test is to get a learning, not a lift. With enough learnings, you can get the real lift,” Dr. Flint McGlaughlin MECLABS
RULES OF THUMB FOR TESTING Things to remember when we are testing • Min. 2000 visitors • Min. a few hundred conversions • At least one full cycle > 2 week • >99% statistical significance • Keep a backlog of test hypothesis • Build next tests as the current one is running *More info at http://kiss.ly/1vF4wUc and http://kiss.ly/1Q4vTm0
Tooltips AND BURGERS
How I like my testing burger Tooltips for your testing framework
Tool Company Why do I like it? Analytics Kissmetrics Shows me my actual customers and their behavior Surveys Qualaroo and/or SurveyMonkey Easy to use/Deep qualitative insights Project Management Trello Manage my time + communicate with collaborators
Tool Company Why do I like it? Company Wiki Confluence Document initial thoughts for everyone Testing platform* Optimizely, Pardot, Adwords Easily build tests and get data Analytics Kissmetrics Let’s me confirm and verify down my funnel Company Wiki Confluence Documentation and learnings for everyone *Includes actual testing platform and tools that have testing capabilities
Not everyone likes their burger the same way How do you take yours?
Trust the process Share your leanings Stack your results A/B TESTING FOR SUCCESS

Publicada em: Marketing
  • FREE TRAINING: "How to Earn a 6-Figure Side-Income Online" ... ♣♣♣ https://bit.ly/2kS5a5J
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Do This Simple 2-Minute Ritual To Loss 1 Pound Of Belly Fat Every 72 Hours ♥♥♥ https://tinyurl.com/bkfitness4u
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui

How to Successfully Run Your First Website A/B Test

  1. 1. `How to Successfully Run Your First Website A/B Test THUE MADSEN
  2. 2. @thuelmadsen @Kissmetrics
  3. 3. The don’ts… The dos… 1 What is A/B testing? Do your homework Repetition Succeed 2 Where do you start testing? TABLE OF CONTENTS 3 A framework for success Tooltips and Burgers4
  4. 4. What is A/B testing? A SCIENTIFIC METHOD OF PROOF
  5. 5. WHAT IS A/B TESTING FOR MARKETING • A scientific method adapted for marketing • Compare two or more versions of an asset to each other • Use data and statistics to determine which version is better • Implement winner and increase conversions
  6. 6. *Image from Optimizely HOW DOES A/B TESTING WORK?
  7. 7. *Image from Kikolani HOW DOES A/B TESTING WORK?
  8. 8. *Image from Dynamicyield HOW DOES A/B TESTING WORK?
  9. 9. SO WHY DO WE A/B TEST? • Lowers risk • Helps us realize the actual impact of proposed changes • Increase learnings • Takes the guess work out of optimization
  10. 10. Where do you start testing? THE DON’TS
  11. 11. *Image from Optimizely WHERE DO YOU START TESTING?
  12. 12. DON’T JUST GUESS
  13. 13. SUB OPTIMIZE • Button colors • Small copy changes • Small layout changes • Fonts, font size, etc. * at least not until you have found 
 a great variation and have lots of traffic
  14. 14. WATCH OUT FOR BUZZ • Don’t just test what the gurus say • What works for others may not work for you
  15. 15. PREMATURE TESTING • Testing without traffic • Before you know what and why you are testing
  16. 16. Where do you start testing? THE DOS
  17. 17. DO YOUR HOMEWORK You have to put in the work to get great results • Learn as much as you can about A/B testing • Study the great people out there • What’s their process and insights?
  18. 18. A framework for success The basis of a successful testing program
  19. 19. Research Know what to test and see the greatest improvements
  20. 20. DIVE INTO YOUR DATA PaidEntered CCAdded to CartViewed Product
  21. 21. FIND YOUR BIGGEST OPPORTUNITIES PaidViewed CheckoutAdded to CartViewed Product
  22. 22. INVESTIGATE THE OPPORTUNITY TO UNDERSTAND IT And/Or
  23. 23. FORM A HYPOTHESIS By doing A, B will happen, Because of C
  24. 24. FORM A HYPOTHESIS By updating branding and increasing form visibility (A), We will get more prospects to register for webinars (B), Because prospect attention will be drawn to the form (C).
  25. 25. BUILD THE TEST
  26. 26. LAUNCH THE TEST VS. Control Variant
  27. 27. THE TEST VS. +24.01% increase in registrations!! 100% Significance
  28. 28. DOCUMENT AND SHARE YOUR FINDINGS
  29. 29. DOCUMENT AND SHARE YOUR FINDINGS Documenting all your tests and results • Increases company transparency • Helps others know where tests are going on • Makes it easier to call tests • Makes learnings available for everyone in company
  30. 30. *Image from Optimizely REPEAT THE PROCESS AS OFTEN AS POSSIBLE
  31. 31. A STORY OF OUR FAILURE VS Control Variant
  32. 32. A STORY OF OUR FAILURE VS
  33. 33. A STORY OF OUR FAILURE VS -13.68% increase in registrations!! 80% Significance
  34. 34. *Quote found on VWO NOT ALL TESTS ARE WINNERS ““Only 1 out of 8 A/B tests have driven significant change.” Noah Kagan AppSumo* ““The goal of a test is to get a learning, not a lift. With enough learnings, you can get the real lift,” Dr. Flint McGlaughlin MECLABS** **Quote found on MarketingSherpa
  35. 35. RULES OF THUMB FOR TESTING Things to remember when we are testing • Min. 2000 visitors • Min. a few hundred conversions • At least one full cycle > 2 week • >99% statistical significance • Keep a backlog of test hypothesis • Build next tests as the current one is running *More info at http://kiss.ly/1vF4wUc and http://kiss.ly/1Q4vTm0
  36. 36. Tooltips AND BURGERS
  37. 37. How I like my testing burger Tooltips for your testing framework
  38. 38. Tool Company Why do I like it? Analytics Kissmetrics Shows me my actual customers and their behavior Surveys Qualaroo and/or SurveyMonkey Easy to use/Deep qualitative insights Project Management Trello Manage my time + communicate with collaborators
  39. 39. Tool Company Why do I like it? Company Wiki Confluence Document initial thoughts for everyone Testing platform* Optimizely, Pardot, Adwords Easily build tests and get data Analytics Kissmetrics Let’s me confirm and verify down my funnel Company Wiki Confluence Documentation and learnings for everyone *Includes actual testing platform and tools that have testing capabilities
  40. 40. Not everyone likes their burger the same way How do you take yours?
  41. 41. Trust the process Share your leanings Stack your results A/B TESTING FOR SUCCESS
  42. 42. KISSMETRICS IS ANALYTICS TO OPTIMIZE YOUR MARKETING www.kissmetrics.com
  43. 43. JUSTIN THORP Senior Marketing Operations Manager, AddThis @thorpus thorp@addthis.com THUE MADSEN Marketing Operations Manager, Kissmetrics @ThueLMadsen tmadsen@kissmetrics.com Questions?
  44. 44. Appsumo reveals its A/B testing secret: only 1 out of 8 tests produce results MORE RESOURCES… A Beginner’s Guide To A/B Testing: An Introduction How to Run AB Tests that Get REAL Results Statistical Significance Does Not Equal Validity (or Why You Get Imaginary Lifts) How Big is the Difference Between Being 90% and 99% Certain? My 7 Rules for A/B Testing That Triple Conversion Rates Optimizely: What is A/B Testing How To Determine The Statistical Significance Of An A/B Test – Basic Introduction

×