Beyond Click Conversions – KISSmetrics’ Director of Marketing Tomasz will cover why it’s so crucial to have a pulse on the entire funnel when A/B Testing. Tomasz will be showing an actual test he ran for this specific demo, showcasing the results from trial conversions, down to those signups that have interacted with the app, to those leads that converted. What you’ll learn: - The influence social traffic has in the funnel - The optimum strategies for A/B testing - How we A/B Test at KISSmetrics - A/B Testing beyond click conversions – having a pulse on the entire funnel and why’s its important

1. PRESENTED BY
+
A/B Testing
You Might Just Be Doing it Wrong

2. +
Tomasz Borys
Director of Marketing,
KISSmetrics
@tbcali
Andrew Caravella
VP of Marketing,
Sprout Social
@andrewcaravella

3. +
Table of Contents
• The optimum strategies for A/B testing
• How we A/B Test at KISSmetrics
• A/B Testing beyond click conversions - having a
pulse on the entire funnel and why it’s important
• The influence social traffic has in the funnel

4. +
The Optimum Strategies for A/B Testing
• Set a Goal
• Baby Steps
• Aim for Statistical Significance
• Never Lose Sight

5. +
Set a Goal

6. +
Baby Steps
People want to
move the needle
and see results fast,
by implementing
multiple changes at once.
But how can
you be sure what
element had an impact?

7. +
Aim for Statistical Significance
SET A GOAL
Aimfor Statistical Significance
getdatadriven.com

8. +
Never Lose
Sight
We can caught up
on what we think is
best to optimize
conversions or
traffic…
but data doesn’t lie

9. +
How We A/B Test at KISSmetrics
• 4,000 and above sample size
• 99% Statistical Confidence
• Blind Eye for 1 Week

10. +
4,000 and Above Sample Size
significant or decent amount of data is needed when
you’re testing beyond click conversions

11. +
4,000 and Above Sample Size
significant or decent amount of data is needed when
you’re testing beyond click conversions
99%
Statistical
Confidence

12. +
Blind Eye for 1 Week
Data can be very erratic the
first several days, so it’s easy to
hit the panic button

13. +
A/B Testing Beyond Click Conversions

14. +
Original

15. +
Variant

16. +
Signups

17. +
Signups

18. +
Signups

19. +
Signup Install JS
Custom
Data
Opps

20. +
Signups
Installed JS

21. +
Signups
Install JS

22. +
Signups
Install JS

23. +
Custom Data

24. +
Custom Data

25. +
Custom Data

26. +
Custom Data
Opportunities Created

27. +
Custom Data
Opportunities Created

28. +
Custom Data
Opportunities Created

29. +
Social Ad Funnel

30. Get a free 30-day trial
sproutsocial.com