Analytics and optimization can each generate great results for businesses. However, it’s at the intersection of analytics and optimization that real value can be extracted. In this session, Analytics Demystified Senior Partner Michele Kiss will share how to better integrate your testing and analytics practices, and real-life examples of success.
28. @michelejkiss #ConvCon
“I suppose it is tempting,
if the only tool you have is a
hammer, to treat everything
as if it were a nail."
Abraham Maslow, “The Psychology of Science” (1966)
33. @michelejkiss #ConvCon
What sells your idea better?
“I believe that …[test idea]”
vs
“I believe that [test idea]…
and with a 2% increase in conversion, that
would drive a $1MM increase in revenue”
34. @michelejkiss #ConvCon
{example} Revenue Forecast
mkiss.me/forecast-impact
• Use existing data
- Traffic, conversion, revenue
• Layer on assumptions
• Estimate impact
36. @michelejkiss #ConvCon
{example} Revenue Forecast
• Current traffic
• Conversion to product pages
• % that view images
• Current conversion rate
• Current conversion rate for image-viewers
37. @michelejkiss #ConvCon
{example} Revenue Forecast
Add your assumptions!
• What % of Product Page and/or Image
Viewers will view Video?
• What % lift in conversion will that drive?
• Any increase in AOV?
• Incremental revenue gain?
41. @michelejkiss #ConvCon
Statistical significance is not enough
“Another way to think of it is that statistical significance isn’t enough.
Large sites … get thousands of conversions per hour, and can see
significance from their testing efforts quite quickly.
But this doesn’t mean they should only run their tests for a few hours.
Rather, they should run their tests long enough to capture a
representative sample of their users over time.”
- Peter Borden, Sum All
44. @michelejkiss #ConvCon
Understand your seasonality
6.0%
6.5%
7.0%
7.5%
8.0%
8.5%
9.0%
9.5%
10.0%
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Year 1
Year 2
Year 3
Year 4
Year 5
66. @michelejkiss #ConvCon
{example} Landing page redesign
Control
vs
RedesignCheck
1 Smith Street, San Jose Check
Check
Check
Check
Performed
-8% worse
(99% Confidence)
81. @michelejkiss #ConvCon
{example} Content Site
Web + Ad Analytics allowed us to value
every.single.page.
• How many (and which ads) are on that page
• How well sold the ads were
• What CPM we were charging
87. @michelejkiss #ConvCon
Targeting your tests to a segment
a/b tests
Simple tests
multivariate/complex
more
complicated
tests
targeted
to analytics
segments
personalization
to individuals
93. @michelejkiss #ConvCon
Go forth and analyse!
ü Find opportunities
ü Estimate the impact
ü Dig in to your results
ü Take your testing further
ü Oh, and befriend your
analysts!