3. Today’s team
Jon Powell
Dr. Flint McGlaughlin
Senior Research
Managing Director
Manager
Funding provided by:
#webclinic
4. Background and test design
Experiment ID: RegOnline Homepage Test
Location: MarketingExperiments Research Library
Test Protocol Number: TP1427
Research Notes:
Background: RegOnline is event management software that lets users create
online registration forms and event websites to manage their events.
Goal: To increase number of completed leads on home page.
Primary research question: Which page will achieve the greatest addressable
lead rate?
Approach: A/B multifactor split test
Funding provided by:
#webclinic
5. Experiment: Control Homepage Step 1
• Testimonials
• Lead form
• Screenshots
5
Funding provided by:
#webclinic
7. Experiment: Treatment Homepage Step 1
• Clarified the Value Proposition
of the offer to emphasize Free
Access.
• Reduced form fields in first step
to capture more partial leads.
• Reduced friction by matching
flow to thought sequence.
7
Funding provided by:
#webclinic
11. Experiment: Results
24.5% Decrease in Conversion
The Treatment generated 24.5% less completed leads
Conversion
Versions Rel. diff
Rate
Control – Two-step home page 2.3% -
Treatment – Three-step home page 1.7% -24.5%
What you needformunderstand: In spite of the control stillvalue and reducing
the amount of
to
fields in the first-step
having clearer
out-performed
the treatment.
11
Funding provided by:
#webclinic
12. Experiment 2: Background and test design
Experiment ID: RegOnline Homepage Test
Location: MarketingExperiments Research Library
Test Protocol Number: TP1428
Research Notes:
Background: RegOnline is event management software that lets users create
online registration forms and event websites to manage their events.
Goal: To increase number of completed leads on home page.
Primary research question: Which process will yield a higher conversion rate?
Approach: A/B multifactor split test
12
Funding provided by:
#webclinic
14. Experiment 2: Results
89.8% Increase in Conversion
The Treatment generated 89.8% more completed leads
Versions Conversion Rate Rel. diff
Control 0.3% -
Treatment 0.5% 89.8%
What you need toofunderstand: Using what we learned in previous tests, we
directed the flow the visitor’s thought sequence through the value
before revealing the price. This is what gave us our lift.
Note: After further testing, we were finally able to increase conversion on
the homepage by 141%
14
Funding provided by:
#webclinic
15. How did we turn this loss into a lift?
F Key Principles
A. The goal of a test is not to get a lift, but rather to get a learning
B. To achieve the maximum amount of learning, your tests should be designed
around two key elements:
1. A research question
15
Funding provided by:
#webclinic
17. Definition: The research question
The Research Question:
A research question is a behavioral question expressed in a factorial
split so it must begin with “which.”
Funding provided by:
#webclinic
18. Examples: The research question
Example #1:
Not this..
What is the best price for product X?
But this…
Which of these three price points is best for product X?
Example #2:
Not this..
Why am I losing customers in the last step of my checkout process?
But this…
Eliminating which form element best reduces customer drop-off?
18
Funding provided by:
#webclinic
19. The research question
? How would you refine the following research questions?
1. What is the best headline for my landing page?
2. How many objectives should I have on my homepage?
19
Funding provided by:
#webclinic
20. What we discovered
F Key Principles
A. The goal of a test is not to get a lift, but rather to get a learning
B. To achieve the maximum amount of learning, your tests should be designed
around two key elements:
1. A research question
2. A theory question
20
Funding provided by:
#webclinic
21. Definition: The theory question
The Theory Question:
The theory question is a broader question about your customer that is
derived from asking “WHY” of the research question.
The answer to the theory question should contribute to a wider
understanding of your customer.
Our researchers call this the Customer Theory.
Funding provided by:
#webclinic
23. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
23
Funding provided by:
#webclinic
24. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you
rather than your competitors?
24
Funding provided by:
#webclinic
25. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you Ideal Customer
rather than your competitors?
25
Funding provided by:
#webclinic
26. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you Ideal Customer
rather than your competitors?
26
Funding provided by:
#webclinic
27. Theory Question: Behavior reflects identity
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you Ideal Customer
rather than your competitors?
27
Funding provided by:
#webclinic
28. Example: The theory question
Research Question Theory Question
Which headline will generate a What does my ideal customer
higher response? prefer?
Why?
A. Save $100 on product X A. Cost savings
B. Product X comes with a B. Product quality
lifetime guarantee
28
Funding provided by:
#webclinic
29. Example: The theory question
Research Question Theory Question
Which call to action will Where is my ideal customer in
generate a higher response? the thought sequence?
Why?
A. Get your product X now A. Ready to buy?
B. Learn more about product X B. Ready to learn more?
29
Funding provided by:
#webclinic
30. Theory Question: How we turned our loss into a win
24%
Homepage
30
Funding provided by:
#webclinic
31. Theory Question: How we turned our loss into a win
Why?
24%
Homepage
31
Funding provided by:
#webclinic
32. Theory Question: How we turned our loss into a win
Why?
24%
Homepage
SEO Landing Page
548%
32
Funding provided by:
#webclinic
33. Theory Question: How we turned our loss into a win
Why?
24%
Homepage
SEO Landing Page
548%
Why?
33
Funding provided by:
#webclinic
34. Theory Question: How we turned our loss into a win
Why?
24% 90%
Homepage Homepage
SEO Landing Page
548%
Why?
34
Funding provided by:
#webclinic
35. Theory Question: How we turned our loss into a win
Why? Why?
24% 90%
Homepage Homepage
SEO Landing Page
548%
Why?
35
Funding provided by:
#webclinic
36. Theory Question: How we turned our loss into a win
Why? Why?
24% 90% 27%
Homepage Homepage
SEO Landing Page
548%
Why?
36
Funding provided by:
#webclinic
37. Theory Question: How we turned our loss into a win
Why? Why?
24% 90% 27%
141%
Homepage Homepage
Cumulative Lift
SEO Landing Page
548%
Why?
37
Funding provided by:
#webclinic
38. The theory question applied
201%
Test results are interpreted and a
second test was created based on
the analyst’s observations
38
Funding provided by:
#webclinic
39. The theory question applied
Again, test results are
interpreted and the next
round of testing is started
for this page
201% 2%
Test results are interpreted and a
second test was created based on
the analyst’s observations
39
Funding provided by:
#webclinic
40. The theory question applied
Again, test results are
interpreted and the next
round of testing is started
for this page
201% 2% 29%
Test results are interpreted and a Test is again interpreted and
second test was created based on transferrable principles are
the analyst’s observations applied to other offer pages
40
Funding provided by:
#webclinic
41. The theory question applied
• The discoveries and insights
about customer motivation 451%
from the three prior tests were
applied to other landing pages
and used to optimize PPC
campaigns.
• The purposeful effort to 302%
identify and selectively apply
these transferrable insights led
to widespread optimization
gains .
257% 28% 603%
41
Funding provided by:
#webclinic
42. Summary: Putting it all together
F Key Principles
A. The goal of a test is not to get a lift, but rather to get a learning
B. To achieve the maximum amount of learning, your tests should be designed
around two key elements:
1. A research question
2. A theory question
42
Funding provided by:
#webclinic
43. Get a Free website grade from HubSpot
Get a FREE detailed report of how your site is
doing compared to your peers and competitors in:
-Generating Traffic
-Search Engine Optimization
-Social Engagement
Visit the link below for more…
Check Out HubSpot’s
Website Grader
bit.ly/wsgrader
43
Funding provided by:
#webclinic
56. View the original presentation free
You can view the full audio
and visual presentation at
MarketingExperiments.com
View the original presentation >>
Funding provided by:
#webclinic
Notas do Editor
2 questions in animation.
Submitted by: JohnPrevious Testing: nonePrimary Objective: To drive traffic into our forms and convert leadsTraffic Source: Organic SEOIntended Audience: Consumers nationwide interested in multiple auto, home and life insurance quotes.
Submitted by: JohnPrevious Testing: nonePrimary Objective: To drive traffic into our forms and convert leadsTraffic Source: Organic SEOIntended Audience: Consumers nationwide interested in multiple auto, home and life insurance quotes.
Submitted by: JohnPrevious Testing: nonePrimary Objective: To drive traffic into our forms and convert leadsTraffic Source: Organic SEOIntended Audience: Consumers nationwide interested in multiple auto, home and life insurance quotes.
Submitted by: MaryPrevious Testing: nonePrimary Objective: To gain people's interestTraffic Source: ad wordsIntended Audience: end users; trades people and retailers
Submitted by: MaryPrevious Testing: nonePrimary Objective: To gain people's interestTraffic Source: ad wordsIntended Audience: end users; trades people and retailers
Submitted by: MaryPrevious Testing: nonePrimary Objective: To gain people's interestTraffic Source: ad wordsIntended Audience: end users; trades people and retailers
Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
Submitted by: WilliamPrevious Testing: nothing structured.Only different text/offer versions from previous events (www.clinical-trials-events.com/ctec.html)Primary Objective: To gain delegate sign ups for conference. free offer with a series of back end sales between now and conference.Traffic Source: mostly email. some ppc.Intended Audience: senior managers/directors of clinical drug development from the pharmaceutical industry
Submitted by: CandacePrevious Testing: none at this timePrimary Objective: To generate purchaseTraffic Source: most from organic searchIntended Audience: Professional foodservice buyers (i.e. restaurateurs, cafeteria mangers, etc)