Brand experience Peoria City Soccer Presentation.pdf
The Nuts and Bolts: How one company implements an entire testing methodology every day
1. How one company implements an entire testing methodology
every day
The Nuts and Bolts
Ryan Hutchings
Director of Marketing
VacationRoost
Daniel Burstein
Director of Editorial Content
MECLABS
2. Session speaker
2
Ryan Hutchings
Director of Marketing
VacationRoost
Hutchings has more than 10 years of online marketing experience. He leads the
marketing team for VacationRoost, a $120 million private company in the travel
industry. From advertising to analytics, he organizes a comprehensive and robust
online marketing strategy focused on tracking and optimization. His specialties
include SEO, PPC, testing, conversion rate optimization, advanced data modeling,
analytics and every technical aspect related to online marketing.
Hutchings is a Google AdWords certified professional, SEOmoz Pro member,
nationally recognized industry speaker and participant of top online marketing
communities. He also help found the SLCSEM organization, which is Utah’s largest
nonprofit group of online marketing professionals.
@HutchSEO
4. About VacationRoost
• 25 websites
• Always running 2 to 3 tests
• 100+ tests run across landing pages, homepages,
lead forms and shopping carts
• Goal of 10% incremental lift in total revenue
generated by testing
8. … to support an iterative optimization cycle
Collecting
data
Planning
experiments
Preparing
treatments
Conducting
tests
Analyzing
results
9. The dual testing track
A/B split test
• Internal system with server-side
splitting, custom variables
• Full manual reporting –
spreadsheets and database
reports
Largetest
Search results
Cart
Products
Testing
Landing page optimization
• Third-party software
• Internal analysis determines
what to test
Smalltest PPC and SEO landing pages
Headlines/CTAs
Images
10. The dual testing track
A/B split test
• Internal system with server-side
splitting, custom variables
• Full manual reporting –
spreadsheets and database
reports
Largetest
Search results
Cart
Products
Testing
Landing page optimization
• Third-party software
• Internal analysis determines
what to test
Smalltest PPC and SEO landing pages
Headlines/CTAs
Images
11. How to test small
Decide Identify Hypothesize Design Analyze
Decide what to test
12. 90%
Replace “our recommendations” section with content snippet
for value proposition banner usage
Restyle “most popular destinations” change hierarchy 40%
13. How to test small
Decide Identify Hypothesize Design Analyze
Identify the target conversion goal
14. What’s a good goal for bounce rates?
> 50%
40%-50% 30%-40%
< 30%
15. What’s a good goal for CTR to the next page
or step?
> 60%
50%-60%30%-50%
< 30%
16. How to test small
Decide Identify Hypothesize Design Analyze
Create a hypothesis
30. ROI template
Incremental Increases
New Bounce Rate
Leads
Sales $
Contribution $
53
$ 40,307
$ 3,225
Incremental Increases
New Bounce Rate
Leads
Sales $
Contribution $
20%
7
5,246 $
420 $
At 10%
At 30%
158
$ 119,256
$ 9,541
25 %
2 d
1,749 $
140 $
31. How do you get the resources to run
large tests?
32. The dual testing track
A/B split test
• Internal system with server-side
splitting, custom variables
• Full manual reporting –
spreadsheets and database
reports
Largetest
Search results
Cart
Products
Testing
Landing page optimization
• Third-party software
• Internal analysis determines
what to test
Smalltest PPC and SEO landing pages
Headlines/CTAs
Images
33. How to test big
Decide Collaborate Identify Create Design Analyze
Decide on an element to optimize
37. How to test big
Decide Collaborate Identify Create Design Analyze
Work with IT to define variables or events
38. Large Testing: Variables are the key
38
Conditions
Segment your users and/or their visits to single or multi-visit conditions.
inlineLeadForm1
mrSearchPage1
Notest
simpleModalLeadForm
simpleModalLeadForm4
control
test
39. How to test big
Decide Collaborate Identify Create Design Analyze
Identify target conversion goal for control
40. The basic funnel
Landing Pages
Search Results
Property
Cart
Billing
Book now, Web lead,
interactions/events
Checkout, add “cross
sell”
Book, fill out info
Bounce
rates, CTR
Purchase
“Rising tide”
A/B
(large test)
Single page LPO
(small test)
41. The basic funnel
Landing Pages
Search Results
Property
Cart
Billing
Book Now, web lead,
interactions/events
Checkout, add “cross
sell”
Book, fill out info
Bounce
rates, CTR
Purchase
Book Now, web lead,
interactions/events
“Rising tide”
A/B
(large test)
Single page LPO
(small test)
10% - 30% minimum targeted gains
60. Setting up your own programmatic testing
1
2
3
4
5
Custom variable tracking system, spreadsheet templates are vital building blocks
Dedicate a person and tie performance to testing
“Top of the funnel” to the “bottom conversion” may have too much noise in
between – keep it simple and isolated
Build a reporting system that ties front end and back end together
Get buy-in from the management team
62. Thank you
Ryan Hutchings
Marketing Director
VacationRoost
@HutchSEO
Daniel Burstein
Director of Editorial Content
MECLABS
@DanielBurstein
Black & White
Headshot
Black & White
Headshot