Experimentation Excellence Webinar Series: Time to scale up

1
Three Stages to
Experimentation
Excellence
Elizabeth Gabster
Kenneth Kutyn

2
Elizabeth Gabster
Lead Value Consultant
Housekeeping: Ask questions in the Q&A widget
We will share the recording and slides
Kenneth Kutyn
Lead Solutions Engineer

3
Getting experimentation started
Experimenting with ease.
Experimenting continuously.
Experimenting across departments.
Experienced but not at the head of the pack (yet).

4
Customers move fast, you need to move faster
Customers today have sky-high expectations.
Digital transformation is ongoing and organizations need to repeatedly optimize
their customer digital experience using experimentation.
Established brands are under threat from digital dominators:
Spotify, Netflix, Amazon, Booking.com, Revolut.

5
Your new ‘business as usual’: The three steps to
get started
Scaling - Number, depth, quality.
Challenging culture.
Shedding legacy technology.

6
get started

7
Scaling - Number, depth, quality

8
Step 1
“Our success at Amazon is a function
of how many experiments we do per
year, per month, per week, per day.”
- Jeff Bezos

9
Where to start?
Where do you want to be? Where do you need to be?
What more can you do? Where can you go?

10
Upping the quality
Look deeper into where the best ideas for experiments come from.
Stay close to your customers and leverage your hard-won
relationships.
A small change can have a massive impact.

11
Upping the quantity and breadth
More experiments means more insights and a greater impact on
the business. More is more.
However, closing the experimentation gap is not easy.

12
Operationalize your experimentation.
Build a team with various skills sets including data
analysis and developers.
Establish a rock solid experimentation pipeline.
Upping the quantity and breadth

13
Develop your hypotheses as fully as possible. Complex
hypotheses means complex, rich data.
Going deeper
Go beyond A/B testing. Save time and create more data.

14
When you’re scaling and speeding up, don’t forget where
you’re heading.
Speed, scale and direction
Moving in the wrong direction is just as bad as not moving at all.
Manage your experimentation team, to extract value from experiments
and direct it.

15
Maximise the output from experiments by connecting data from any
and every source.
Get all the data you can
Measure the impact of experiments across almost any metric.
“Understand the entire customer journey
by measuring the impact of our
experiments across all digital touchpoints ”
- Just Eat

16
Encouragement from management raises the perceived value
of experimentation.
Digital experimentation as business experimentation
Connect every experiment to business strategy, strategic
goals and the bottom line.
Prove experimentation is a cash-positive part of the business.

17
Keep your business competitive with digital experimentation.
Survival of the fittest
Competitors are still only one click away and always improving too.
If you’re not moving forwards, you’re moving backwards. Don’t
get squeezed out of the market.

18
get started

19
get started

21
Great digital experimentation is not just down to great people.
Great technology is needed too.
Don’t get held back
Legacy technology suites can be an obstacle.
Don’t let anything keep you from going from good to great.
Shedding legacy technology

22
The keys to greatness
55% of experimenters going from good to great want to
automate more of the experimentation process.
This is key to scaling and speeding up.
You need an all-in-one technology stack. But, also one with
sophisticated solutions to complex needs.

23
The good to great wishlist
Export in-depth statistics for rich data.
Machine learning tools for real-time analysis.
Automation for speed.

24
Technology stack dream team
Marketing
Product
Collaboration
Monitoring
Cloud infrastructure

25
Marketing
Product
Collaboration
Monitoring
TEALIUM + CONTENTFUL

26
Marketing
Product
Collaboration
Monitoring
AMPLITUDE + FULLSTORY

27
Marketing
Product
Collaboration
Monitoring
ATLASSIAN

28
Marketing
Product
Collaboration
Monitoring
ATLASSIAN
NEW RELIC

29
Marketing
Product
Collaboration
Monitoring
ATLASSIAN
NEW RELIC
AWS

30
Technology which allows you to remove the guesswork can
amplify your human talent and expertise.
The right technology stack can level-up the quality, quantity of
experiments and deliver rich data at lightning speeds.
Faster action keeps you competitive and gives the best return
on investment.

31
get started

32
get started

34
You’re not getting started anymore, it’s time to step up from
being a cultural upstart.
Now is the time
Snowball, pick up momentum and win over the majority of
your business.
Keep transitioning from the traditional top-down to the more
innovative bottom-up culture.
Challenging culture

35
Management can fan the flames of digital experimentation.
Management styles for experimentation
More opportunities means more valuable data and insights
to improve the business.
Two management styles:
• Peer review
• Top metrics
Challenging culture

36
It won’t discourage experimentation.
Management style 1: Peer review
More hoops means more thought. More thought means
greater complexity and rationale.
Experiments become more valuable. Netflix and AirBnb’s
adoption shows it ca be successful.
Challenging culture

37
Monitor business performance from revenue and user acquisition to
catch checkout rates and more.
Management style 2: Top metrics
Communicate which metrics are most important.
Focus experimentation where it is needed most.
Challenging culture

38
Siloed business functions miss out on:
• The news of success.
• Experimentation terminology.
• Valuable insights being discovered which they could implement.
Breaking down silo walls
Communicate effectively. Try distributing a monthly experimentation
newsletter.
Challenging culture

39
Good to great summary
Scale the number, depth and quality of
digital experimentation.
Challenge your organization's culture.
Invest in shedding legacy technology.

40
Download the report.
Stay for Q&A.

41
In the next webinar:
Revitalising hypotheses and the digital experimentation engine.
How to embed digital experimentation across
an entire business.
• Scope
• Function

42
You’ve got questions,
we’ve got answers

Experimentation Excellence Webinar Series: Time to scale up

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Experimentation Excellence Webinar Series: Time to scale up

Semelhante a Experimentation Excellence Webinar Series: Time to scale up (20)

Mais de Optimizely

Mais de Optimizely (20)

Último

Último (20)

Experimentation Excellence Webinar Series: Time to scale up