1.
Innovate Faster with the
Modern Growth Stack
Brion Hickey
Growth
Cisco
Principal Product Manager
Amplitude
Carter Foxgrover
Senior Product Marketer
Optimizely
David Isquick

2.
Brion Hickey
Growth
Cisco
Principal Product Manager
Amplitude
Carter Foxgrover
Senior Product Marketer
Optimizely
David Isquick

3.
• We are recording today’s presentation
• Slides will be available after the webinar is complete
• Please submit questions via the text box
Housekeeping

4.
Growth - Tech Stack

5.
About Me
Over 120 companies worldwide
Then Now

6.
USA

7.
USA EMEA

8.
USA EMEAAPAC

9.
Came home.
Got a cat.
Named him
Meatball.

10.
Optimization > Growth
Test stuff > Knowing your end users
Transitional Period

11.
Growth
Philosophy

12.
Growth
Philosophy
Strategy

13.
Growth
Philosophy
Strategy
Technologies

14.
Growth
Philosophy
Strategy
Technologies
Tactics

15.
Growth
Philosophy
Strategy
Technologies
Tactics

16.
Growth Tools
Customer Journey

17.
Marketing Analytics (auditing)
Customer Journey
Growth Tools

18.
Product Analytics / BehaviorsMarketing Analytics (auditing)
Customer Journey
Growth Tools

19.
Product Analytics / BehaviorsMarketing Analytics (auditing)
Experimentation framework
Customer Journey
Growth Tools

20.
Product Analytics / BehaviorsMarketing Analytics (auditing)
Experimentation framework
User Testing
Customer Journey
Growth Tools

21.
Marketing Analytics

22.
Marketing Analytics

23.
“It was an awful
integration”
Marketing Analytics

24.
“It was an awful
integration”
“The reports can’t
be right”
Marketing Analytics

25.
“It was an awful
integration”
“The reports can’t
be right”
“The pixel isn’t
firing”
Marketing Analytics

26.
“It was an awful
integration”
“The reports can’t
be right”
“The pixel isn’t
firing”
“I don’t trust
(marketing
analytics
company
here)”
Marketing Analytics

27.
ObservePoint

28.
Auditing Marketing Analytics
● “It was an awful integration”
○ Specifically addressing the integrity of
the integration
○ What tags are missing
○ What pages need to be fixed

29.
Auditing Marketing Analytics
● “The reports can’t be right”
● “The Pixel isn’t firing”
○ Complete inventory of tags firing
(working as expected)
○ List of requests sent for each page
○ Duplicate tags, requests

30.
Auditing Marketing Analytics
● “I don’t trust (insert any marketing analytics
company here)”
○ Output of a initial score/ grade
○ Comprehensive list of elements to fix
○ Update score, gain trust

31.
In-Product Behavior Analytics

32.
Product Analytics / BehaviorsMarketing Analytics (auditing)
Customer Journey

33.
Amplitude

34.
Text Here
Amplitude
Retention
North
Star
Metrics

35.
Text Here
Amplitude
Retention
A ha
Moment
North
Star
Metrics

36.
Text Here
Amplitude
Retention
A ha
Moment
Product
Market
Fit
North
Star
Metrics

37.
Text Here
Amplitude
Retention
A ha
Moment
Product
Market
Fit
North
Star
Metrics

38.
Text Here
Amplitude
Retention
A ha
Moment
Product
Market
Fit
North
Star
MetricsWhat to
A/B
Test

39.
But wait...

40.
Real End User Testing

41.
Product Analytics / BehaviorsMarketing Analytics (auditing)
User Testing
Customer Journey

42.
Usertest.io

43.
User Testing
● It’s not expensive anymore / $15
● Reporting drop off. Why?
● Initiated user testing.
● Our buttons didn’t work.
○ Wait what?
○ Ad blockers

44.
User Testing
● It’s not expensive anymore / $15
● Reporting drop off. Why?
● Initiated user testing.
● Our buttons didn’t work.
○ Wait what?
○ Ad blockers
Dude, you had 1 job :|

45.
User Testing
● Turn it into a working cadence
○ Weekly, monthly
○ Stay consistent
○ Especially before and after redesigns/feature releases
● It take a lot of time, so what
○ Create summary video
○ Invaluable feedback

46.
Get Organized
● Start to document findings
● Access to content isn’t hard
● Easy for engineers to consume
○ Screenshots
○ Behaviors
○ Time stamp
○ How to reproduce
○ Expected behaviors

47.
Optimizely

48.
Marketing
Customer Journey
Optimizely

49.
ProductMarketing
Customer Journey
Optimizely

50.
ProductMarketing
Experimentation framework
Customer Journey
Code Level (Full Stack)
Optimizely

51.
ProductMarketing
Experimentation framework
Customer Journey
Code Level (Full Stack)
Personalization Recommendations Mobile Native
Optimizely

52.
I can change

53.
I can change I’m listening

54.
I did this
I did this

55.
Product Analytics / BehaviorsMarketing Analytics (auditing)
Experimentation framework
User Testing
Customer Journey
Growth Tools

56.
Stay In Touch
Brion.tw
Brionh@gmail.com

57.
Build Better
Products
Account Owner | email_id@amplitude.com | phone number
Product Analytics
for The Modern
Growth Stack

58.
Corporate Background
AMPLITUDE
● Launched Jan 2014
● Backed by Benchmark, Battery & IVP
● Now power over 20k products for over 5k
companies across the globe
● Analyzed over 2.5T user actions
● #1 Retention Playbook in industry
● Leader in self-serve Product Analytics

59.
The Fastest Moving Growth & Product Teams use Amplitude
OUR CUSTOMERS
Consumer Media B2B Ecommerce Finance Healthcare

60.
Product Analytics
for Growth
2

61.
Science, not Art
Quarterly releases and NPS surveys
aren’t enough. Modern growth teams
run like optimization machines:
● Mine data for insights
● Rapidly test ideas for impact
● Improve experiences every day
ADAPT OR DIE

62.
Growth Teams Need 3 Types of Product Insights to Innovate
WHAT IS PRODUCT ANALYTICS FOR GROWTH?
How do
CUSTOMERS
use our
products?
Examples
Do some users binge watch shows? If so which ones? What
shows are more likely to be binge watched?

63.
Growth Teams Need 3 Types of Product Insights to Innovate
WHAT IS PRODUCT ANALYTICS FOR GROWTH?
How do
CUSTOMERS
use our
products?
What
FEATURES
do we prioritize
right now?
Examples
Do some users binge watch shows? If so which ones? What
shows are more likely to be binge watched?
Which features increased IBM Cloud Usage and revenue
amongst our customers? How can we double down on them?

64.
Growth Teams Need 3 Types of Product Insights to Innovate
WHAT IS PRODUCT ANALYTICS FOR GROWTH?
How do
CUSTOMERS
use our
products?
What
FEATURES
do we prioritize
right now?
How are our
METRICS
improving?
Examples
Do some users binge watch shows? If so which ones? What
shows are more likely to be binge watched?
Did purchase frequency increase amongst shoppers who
received coupon code notifications last week?
Which features increased IBM Cloud Usage and revenue
amongst our customers? How can we double down on them?

65.
Most Tools are not Designed to Correlate Product & Impact
WHAT IS PRODUCT ANALYTICS FOR GROWTH?
Product Engineering InfraMarketingBusiness Intelligence
Without Amplitude, growth teams have to rely solely on analysts/ data scientists to answer complex
questions

66.
Relying only on Analysts Creates Data Breadlines
WHAT IS PRODUCT ANALYTICS FOR GROWTH?
BI and
Query Tools
Production
Database
Growth
Marketing
Executives
Analysts
Sales
Finance
Without Amplitude, Growth Teams can’t explore user behavior on their own and spin cycles waiting
for analytics facetime
4-8 weeks delay

67.
Why Amplitude
3

68.
Amplitude is your Growth Team’s Personal Data Assistant
GROWTH/ PRODUCT DATA SCIENCE AS A SERVICE
Product Growth
Execs
Sales Mrkting
Analysts
Data
Scientists
Engineers
Growth Teams can investigate and explore real time data for 90% of their needs while also sharing their
insights with the entire organization
Same day answers

69.
Amplitude has unrivalled Depth of Behavioral Analytics
THREE LEVELS OF NEW AGE ANALYTICS
LEVEL 1: KPI Monitoring
How many people are using X feature?
LEVEL 2: Problem Diagnosis
What is the drop-off of our
onboarding flow?
LEVEL 3: Behavioral Analysis
What user behaviors are causing
feature adoption?

70.
VP Growth @

71.
Case Study: Intuit
QuickBooks
4

72.
Intuit QuickBooks has become their fastest growing unit
MINI CASE STUDY: 3 BUSINESS UNITS AT INTUIT NOW USE AMPLITUDE

73.
Intuit team initially had poor KPIs for their invoicing product
MINI CASE STUDY: HOW QUICKBOOKS TEAM LAUNCHED “CONNECT”
How do
CUSTOMERS
use our
products?
Customers were creating invoices in Quickbooks but not
emailing them. Fewer invoices were getting paid
Behavior
Problem

74.
Their product team quickly diagnosed the behavior issue
MINI CASE STUDY: HOW QUICKBOOKS TEAM LAUNCHED “CONNECT”
How do
CUSTOMERS
use our
products?
What
FEATURES
do we prioritize
right now?
Amplitude helped them discover that this was because
customers who preferred invoicing through Gmail were not
able to get value from Quickbooks. Gmail has >1B DAUs so
they prioritized a Quickbooks Gmail plugin
Customers were creating invoices in Quickbooks but not
emailing them. Few invoices were getting paid on time.
Feature
Insight
Behavior
Problem

75.
They launched their new feature “Connect” w/ great impact
MINI CASE STUDY: HOW QUICKBOOKS TEAM LAUNCHED “CONNECT”
How do
CUSTOMERS
use our
products?
What
FEATURES
do we prioritize
right now?
How are our
METRICS
improving?
Quickbooks Gmail has dramatically increased # invoices paid
on time and decreased # clicks needed to get paid opening up
a new market opportunity for Intuit
Customers were creating invoices in Quickbooks but not
emailing them. Fewer invoices were getting paid
Feature
Insight
Resolution
and Impact
Behavior
Problem
Amplitude helped them discover that this was because
customers who preferred invoicing through Gmail were not
able to get value from Quickbooks. Gmail has >1B DAUs so
they prioritized a Quickbooks Gmail plugin

76.
Intuit has built a data driven culture for Growth
MINI CASE STUDY: 3 BUSINESS UNITS AT INTUIT NOW USE AMPLITUDE
Connect: Intuit’s Fastest Growing Product

77.
Optimizely

78.
Virtuous Cycle
Analytics +
Experimentation
Senior Product Marketer
Optimizely
David Isquick

79.
Everyone plays a part in growth
Driving Awareness of
Those Products
Building Great
Products and
Experiences
Understanding Product
and Marketing
Performance
Improving Product
and Marketing
Performance
Product-driven Marketing-Driven Analytics and
Research Driven
Experimentation-Driven
Growth

80.
Enable Rapid Experimentation by Everyone
Built for business teams
See results in real-time
Act with confidence
and product
development teams
Always-on
experimentation

81.
Analyze
HypothesizeExperiment
Virtuous Cycle
Analytics meets
Experimentation

82.
Analyze
HypothesizeExperiment
Virtuous Cycle
Analytics meets
Experimentation
• Core Metrics
• Funnels
• Retention
• LTV

83.
Analyze
HypothesizeExperiment
Virtuous Cycle
Analytics meets
Experimentation
• Core Metrics
• Funnels
• Retention
• LTV
• Product
improvements
• Marketing
improvements

84.
Analyze
HypothesizeExperiment
Virtuous Cycle
Analytics meets
Experimentation
• Website A/B tests
• Product
experiments
• Core Metrics
• Funnels
• Retention
• LTV
• Product
improvements
• Marketing
improvements

85.
Analyze
HypothesizeExperiment
Virtuous Cycle
In action
Analyze booking
conversion funnel

86.
Hypothesis: Adding More Context Will Help Drive More Bookings

87.
Analyze
HypothesizeExperiment
Virtuous Cycle
In action
Analyze booking
conversion funnel
Adding a step to
funnel to provide
more context will
increase
conversion

88.
Analyze
HypothesizeExperiment
Virtuous Cycle
In action
Analyze booking
conversion funnel
Split traffic and
analyze initial
results in terms of
conversion
Adding a step to
funnel to provide
more context will
increase
conversion

89.
Increase revenue with counterintuitive changes
Adding a step to the booking funnel produced a 5% increase in
revenue

90.
Questions?