Mais conteúdo relacionado Semelhante a Leading the product 2017 - Susan Teschner - Speaker Slides (20) Mais de Brainmates Pty Limited (20) Leading the product 2017 - Susan Teschner - Speaker Slides1. Leading The Product 2017
Speaker Slides
Sydney, Australia
Susan Teschner
99designs
For more information go to
www.leadingtheproduct.com
2. © 99designs, Inc.© 99designs, Inc.
Experimenting
with
ExperimentationPro tips for doing it well
by Susan Teschner
5. © 99designs, Inc. 5
Manage products.
Manage people
managing
products.
by Evilltimmby LanaD.
6. 6
World’s largest marketplace
for graphic design
We’re very busy
—
450,000happy customers
We’re famous but with
hidden talents
—
36
Design categories
by Vladson
We’re home grown and
a bit grown up
—
2008
Founded in Melbourne
115
Fabulous people
3
Offices (BER, OAK, MEL)
$200mpaid to designers
8. © 99designs, Inc. 8
“doing a thing to discover something
unknown or to test a principle”
— poorly paraphrased from dictionary.com
19. © 99designs, Inc.Commercial in confidence 19
Start off on the right foot
—
▪ clearly stated hypothesis
▪ definition of success
▪ specific metrics
by ktoons
20. © 99designs, Inc. 20
Beware the dragon of
statistical significance
—
by ilustreishon
21. © 99designs, Inc. 21
Here’s how
—
https://unbounce.com/ab-test-duration-calculator/
Sample Size & Test Duration Calculator
Current conversion rate (%)
Desired lift (%)
Number of variations
Average daily visitors
To achieve valid test results*, you need:
A total sample of (users):
Test duration of (days**):
10800
5
10
20
3
2000
22. © 99designs, Inc.
Sample Size & Test Duration Calculator
Current conversion rate (%)
Desired lift (%)
Number of variations
Average daily visitors
To achieve valid test results*, you need:
A total sample of (users):
Test duration of (days**):
10800
54
10
20
3
200
Sample Size & Test Duration Calculator
Current conversion rate (%)
Desired lift (%)
Number of variations
Average daily visitors
To achieve valid test results*, you need:
A total sample of (users):
Test duration of (days**):
10800
5
10
20
3
2000
22https://unbounce.com/ab-test-duration-calculator/
Here’s how
—
23. © 99designs, Inc. 23
This was a bit too
long to wait
—
How long will your experiment take?
How many eligible visits do you have per day?
What percentage of visits will see the change?
What is your current conversion rate?
How will you change conversion?
How confident do you want to be of this?
How likely should you be to detect the change?
1,400
50%
2%
5%
95%
90%
You should run this experiment for 1 year, 7 months and 28 days.
29. © 99designs, Inc. 29
Beware the gold plating
—
by MattDyckStudios
https://www.quora.com/How-is-pretotyping-different-from-prototyping
32. © 99designs, Inc.© 99designs, Inc. 32
“We increased
sales by 37%
and all we had
to do was
change the
color of a
button!
by Pierre d’Arterie
42. © 99designs, Inc. 42
… or take your chances
with the people who look
after it
—
by Thedaydreamer
46. © 99designs, Inc. 46
A story about Social Sharing
by miljanristic by FRITO-DESIGNstudio
by Joedsign
49. © 99designs, Inc.
Until one
day…
—
Ashish
Yep, I’ve also flagged this feature with @Miranda Burford as
something we should critically evaluate as I’m concerned the
benefits have not outweighed the costs.
Susan
Good call @Miranda Burford. And it didn’t go unnoticed that we’re
the only ones who ticked off our To-Do’s.
Miranda
Just to circle back on this, we reviewed Social Sharing and made
the decision to retire it. If you’re interested in how we came to that
decision, you can check out the slide deck.
52. © 99designs, Inc.
2017-05-17
2017-07-05
2017-05-12
2017-04-18
Start End Team
2017-04-29
Growth
Activation
Growth
-2% click through
-11% reduction of click through to
Other design category
50%
2017-07-17 -10% clicks
0% conversion impact
Growth
Acquisition
/categories
Email
forward in
brief
Improved
search on
categories
Spruced up
cross sell
Experiment Page
Conversion
goal
/work
Click to
product
pages
Click
repeat
purchase
Visual brief Increase
email
capture
Minimum /
Aimed
improvement
5%
65%
33%
Observed
improvement
52
At a glance
—