Matt presents the four types of test every Ecommerce business should be running. He explains how to identify what to test and how to prioritise your testing schedule, as well as settling once and for all the question “what’s wrong with testing button colours”? He’ll cover pricing and price framing, proposition testing and radical redesigns to optimise for engagement and conversion. Key take-aways for each section are shared ready for businesses to go and implement in their own organisation.
1. YOU’RE TESTING
WHAT?!
FOUR TYPES OF TESTS
YOU SHOULD BE
RUNNING
MATT LACEY
Head of Optimisation
@matt_lacey12
27TH MARCH 2014
CAMP DIGITAL
MANCHESTER
2. @matt_lacey12
#CampDigital
Head of Optimisation at PRWD
One of the UK’s most respected CRO agencies
Lead Optimisation Programmes for The North
Face, Student Room & Sliderobes
Experienced in Web Analytics, User
Researcher & A/B Testing
4. @matt_lacey12
#CampDigital
What should we be testing?
Pricing & Price Framing
Proposition Testing
Optimising for Engagement
Optimising for Conversion
TALK
OVERVIEW
6. @matt_lacey12
#CampDigital
WHAT’S
WRONG WITH
TESTING
BUTTON
COLOURS
Disclosure: We test buttons. Size,
Prominence, Copy, Positioning and yes,
Colour.
But:
There are likely to be constraints that
limit the number of tests that you can run
based on design & developer resource
In most situations we find that there are
tests that can be run that will deliver
more interesting results and more value
8. @matt_lacey12
#CampDigital
SECTION TITLE
WHAT DOES IT
INVOLVE?
Analytics
User Research
Cross Device User Testing
Hypothesis Development
Concept Development
Psychology & Persuasion techniques
Usability Best Practice
Testing Strategy and Planning
Test Setup & QA
Test Results Analysis
9. @matt_lacey12
#CampDigital
What is the data telling me about visitor
behaviour?
• Top landing pages
• Pages with high traffic
• Key leak or conversion points
What are users telling me about their
experiences? What can we observe about
their behaviour?
• Usability errors
• Influential proposition messaging
What does the business know about
customer struggles or business challenges?
• Customer service insights
• Merchandising team
HOW TO
CHOOSE WHAT
TO TEST
10. @matt_lacey12
#CampDigital
Triage
• Just Do It – JDI
• Single Feature Test
• Batch Testing
• Radical Redesign Testing
• Larger Redesign projects
Triangulation
• Multiple research methods confirm or
strengthen the same issues
Potential Impact vs Effort Required
• Likelihood to impact key metrics
vs
• Design effort
• Development effort
• Development schedule
HOW TO
PRIORITISE
YOUR TESTS
12. @matt_lacey12
#CampDigital
Through moderating or observing hundreds
of user testing sessions we have often
asked:
WHY TEST
PRICING?
“If you could buy the
same product from two
different websites, what
factors would be
important in helping
you choose which to
buy from?”
User consistently respond with:
1. Price
2. Delivery (price and timescales)
13. @matt_lacey12
#CampDigital
1. Pricing should be based on value, not
cost
2. Prices should be tangible
3. Prices should be comparable
4. To change your prices, you must reframe
5. Price differentiation is the key enabler
of profit
6. Pricing communication shapes the
customers perception of value
7. You must be prepared to lose some
sales.
Source: Leigh Caldwell:The Psychology of Price: How to use price to increase demand,
profit and customer satisfaction
SEVEN
PRINCIPLES OF
PRICING
14. @matt_lacey12
#CampDigital
Three Principles of Anchoring
1. The perception of price is often based
on the context.
2. Decision making and the power of
suggestion
3. Avoiding extremes
Source: Price Intelligently
http://blog.priceintelligently.com/blog/bid/181199/Price-Anchoring-to-
Optimize-Your-Pricing-Strategy
TECHNIQUE:
PRICE
ANCHORING
15. @matt_lacey12
#CampDigital
Four Options:
1. Control
2. Variation A – (SAVE £22)
3. Variation B – (SAVE 38%)
4. Variation C – remove ‘(cancel
anytime)’ from monthly
WHICH TEST
WON?
12.5%overall increase in
subscriptions for Variation A (SAVE £22)
16. @matt_lacey12
#CampDigital
1. Control - Annual £39.99
(Save £44)
2. Variation – £3.33 per
month (£39.99 annually)
WHICH TEST
WON?
77%overall increase in annual
subscriptions for the Variation
17. @matt_lacey12
#CampDigital
Consider the range of price options
available and test
• This could apply to product or service
costs, delivery costs, etc
Run tests to make price more transparent
• What do people get for their money
• Test messaging that explains what people
are paying for
Control comparisons
• Test using competitor prices for
comparisons
PRICE & PRICE
FRAMING
TAKEAWAYS
19. @matt_lacey12
#CampDigital
Watch clip: http://www.youtube.com/watch?feature=player_embedded&v=ps7bB9-ksMY
“
The second a visitor comes to you they have
in mind: “Why should I buy from you versus
all your competitors?”
You need to display clear unique value
propositions or unique campaign
propositions.
Everyone who is successful in online
conversion is doing this. If you’re not, you’re
wasting your time.
Bryan Eisenberg,
Godfather of CRO
20. @matt_lacey12
#CampDigital
We can define this as 'Why should I buy
from you'. Typically this is a short
statement that is displayed prominently on
the site.
Here are a few examples:
• ASOS:
"Discover fashion online”
• Naked Wines:
"We are a customer funded wines business"
UNIQUE VALUE
PROPOSITION
(UVP)
22. @matt_lacey12
#CampDigital
USPs - Unique Selling Proposition
USPs are more focused on the shopping
experience and what advantages visitors
gain from shopping with you.
Examples:
• Free next day delivery over £50
• 365 Day Returns
• The Official Online Store
• The UK's largest stockist
UNIQUE
SELLING
PROPOSITIONS
(USPS)
25. @matt_lacey12
#CampDigital
Unique Campaign Propositions -
UCP’s are messages based on specific
campaigns or promotions. These messages
will often be time limited.
Examples:
• Last Christmas Deliveries on 18th Dec
• XX% off this weekend only
• Free next day delivery all week
UNIQUE
CAMPAIGN
PROPOSITIONS
30. @matt_lacey12
#CampDigital
GET REVISING
SIGN-UP TABLE
“
Price doesn’t put me off, although the
basic package is more than enough for
me.
I think the site has already got
everything I need.
Moderated User testing participants
32. @matt_lacey12
#CampDigital
GET REVISING
SIGN-UP TABLE
185%increase in premium subscriptions
with a small drop in free subscription. This was seen to
be a good trade-off.
You have more features held back to
encourage people to subscribe at a later date.
33. @matt_lacey12
#CampDigital
Test what features/functionality is given
for free and what is paid for
Test how your proposition is displayed/
framed on your site
Test offering different thresholds for free
delivery
Test introducing urgency to the process
PROPOSITION
TESTING
TAKEAWAYS
35. @matt_lacey12
#CampDigital
This opens up the top of the conversion
funnel.
This is likely to be the point where you have
the largest number of potential test
participants.
Search for landing or key site pages with
high bounce rates or those that rarely lead
to conversions
Key Areas to Test
• Homepage
• Top landing pages
• Brand sites/Brand content
OPTIMISING FOR
ENGAGEMENT
41. @matt_lacey12
#CampDigital
Let users know that they have landed in the
right place
Make sure that pages have a clear focus
and visual hierarchy. Users need to know
where to begin.
Test both simplifying the experience and
adding in more detail
A smaller amount of highly qualified
visitors can trump large amounts of un-
qualified visitors and convert better
ENGAGEMENT
OPTIMISATION
TAKEAWAYS
43. @matt_lacey12
#CampDigital
Testing towards the end of your funnel
helps to convert users who are closest to
converting and tend to have higher intent.
Improvements here opens up the bottom of
the conversion funnel.
Simplifying and removing issues that slow
momentum is crucial.
Key Areas to Test
• Checkout (login, delivery, payment)
• Contact/Sign-up forms
OPTIMISING FOR
CONVERSIONS
47. @matt_lacey12
#CampDigital
Don’t be afraid to offer visitors an
opportunity to convert at an early stage
Use persuasive techniques to pull users
through (not push!)
Simplify forms to the bare minimum fields
required
Reduce errors. There’s a huge correlation
between users getting an error and
abandoning the funnel
Provide key supporting proposition
messaging throughout the process
Re-enforce positive messages (e.g. security
in checkout)
CONVERSION
OPTIMISATION
TAKEAWAYS
48. @matt_lacey12
#CampDigital
With the right kind of research
and preparation the odds are
stacked in your favour.
Therefore:
Be data driven. Be bold.
Medium risk, high
reward.
FINAL
TAKEAWAY
49. @matt_lacey12
#CampDigital
Conversion optimisation case studies
Usability best practice
User research insights
CRO presentation slides
Persuasive design slides
Research & optimisation tools
One final resources link...
bit.ly/CROresources