Presentation for the Founder Institute mentoring session.

1. Building Products
Quantitatively in SaaS
Businesses
Soren Harner, Bigcommerce
© 2003 – 2013 Bigcommerce Pty. Ltd.
Wednesday, 6 November 13

2. About me
2012 VP Engineering
Stage 2 Startup
Venture-backed
$35MM
2010 Founder & Datavore
Stage 1 Startup
Bookstrapped
2007 VP Engineering
Stage 2 Startup
2000 Search & Analytics,
Stage 2 Startup
Stage 3 Acquisition
Venture-backed > $80MM
Director Engineering
Wednesday, 6 November 13

3. Hill Climbing
Wednesday, 6 November 13

4. SaaS Business Levers
•
•
•
•
•
•
Cost to acquire customers (CAC)
Average Revenue per User (ARPU)
Churn
•
Goal: Negative revenue
retention (upgrades)
Conversion (CVN) by channel
•
Payback
Life time value (LTV)
•
•
LTV = ARPU/Churn
LTV / CAC > 1 is success!
Monthly Recurring Revenue (MRR)
•
Magic Number
Wednesday, 6 November 13
http://blog.asmartbear.com/coc-cancellation-saas-metric.html

5. LTV / CAC
Model
“successful”
1.2
0.9
0.6
0.3
0
Q1-10
Q2-10
Q3-10
Q4-10
Q1-11
Q2-11
Q3-11
Q4-11
To IPO range is maybe 4 to 6 LTV / CAC today
Fine print: Like all metrics use to track, not as strategy
Wednesday, 6 November 13

6. Tested hundreds of versions
Wednesday, 6 November 13

7. Signup, upgrade, onboarding
• Invest in a great experience
• Test with usertesting.com
• Track the funnel:
• Number of trials
• Leads to Trials to Paid by weekly cohort
• Onboarding success
Wednesday, 6 November 13

8. Product UX Testing
Numbers don’t tell you how to improve.
Workshops
(personas)
Concepting
(sketches)
Prototypes
Qualitative
(Axure)
You can use an iPad at a Cafe and show keynote.
Key point is to test mockups on people first.
Wednesday, 6 November 13
Quantitative
(Loop11)

9. Feedback
Wednesday, 6 November 13

10. Measure Product Success
•
Customer success
•
•
•
•
•
Wednesday, 6 November 13
Time to first sale
Time to launch
Trend monthly or weekly cohorts
Engagement score during trial
Churn and conversion predictive models

11. Feature KPI
•
Before you build a
feature, define a success
KPI
•
•
Wednesday, 6 November 13
e.g. Number of
product reviews
completed
e.g. 30% fewer
abandoned carts
•
e.g. completing task
more quickly
•
Add instrumentation
before and after
•
Deploy to subset of
customers to test
•
Consider Mixpanel

12. Split Test Important Things
• Persona analysis and customer interviews
on new brand
• Route traffic p new, 1-p old (control)
• Measure effect on conversion with 95%
confidence interval of a difference and
ZScore
Wednesday, 6 November 13

13. A/B Testing Report and significance
p +/- (1.65 x SE)
https://developer.amazon.com/sdk/ab-testing/reference/ab-math.html
Wednesday, 6 November 13

14. Network Diffusion
Atlassian JIRA to OS communities.
SEO / SEM limitations — budget and effectiveness.
Wednesday, 6 November 13

15. Power law
Think 80/20 rule.
Store revenue, traffic, disk usage, number products, etc.
http://www.hpl.hp.com/research/idl/papers/ranking/ranking.html
Wednesday, 6 November 13

16. Continuous Delivery
• Feature flag everything and gently deploy
• Deploy daily — huge difference!
• Read 12factor.net (dev, staging, production)
• Monitor (i.e. NewRelic)
• Practice BDD & continuous integration
• Always have a recovery plan
Wednesday, 6 November 13

17. Ask permission, get feedback
Wednesday, 6 November 13

18. User Stories
Wednesday, 6 November 13

19. Commutative flow
Wednesday, 6 November 13

20. Startup Friendly Stack
•
Use Ruby Sinatra (or Clojure) +
CoffeeScript + Backbone +
Handlebars on Heroku
•
Nesta for CMS and learn textmate,
Haml & Sass, see https://github.com/
sharner/Workshop-Site
•
HTML5 mobile first, responsive
design
•
Use Mailchimp, Wufoo,
SurveyMonkey
•
R for data (http://r-project.org) —
treat as code, put in Github
•
Log events to understand customer
success (see Heroku Plugins)
Wednesday, 6 November 13

21. Peopleware
•
Do learn to code simple website
changes and growth hacking
•
Learn some stats
•
Do find a technical co-founder
•
Don’t offshore
•
Don’t churn people
•
Do find designer who codes
frontend and does user testing
Wednesday, 6 November 13