Results of the Lean Startup Research survey as one part of an Executive MBA master thesis in the area of early stage high-tech entrepreneurship. You can buy the complete thesis with all the results here: https://www.facebook.com/ideasenabled/app_190322544333196

1. The Lean Startup Research Project
Results of the Lean Startup Approach survey
Christian Kählig – Executive MBA in Innovation and Business Creation (TUM, HHL, UC Berkeley)
FACEBOOK: @ideasenabled -- LINKEDIN: http://de.linkedin.com/in/ckaehlig -- WEB: ideasenabled.com
May 10, 2011

2. Preface
• The following report contains elements of the research conducted as part
of a three month Executive MBA master thesis with the following topic
Facilitating Opportunity Development:
Increasing Understanding of the
Lean Startup Approach in
Early Stage High-Tech Entrepreneurship
• The focus of this presentation is on the Lean Startup Approach survey
which did not strive for statistical significance but qualitative data.
• The presentation is primarily designed to be read which is why it contains
more text than recommended for an in-person presentation.
• In case you are interested in the complete Lean Startup research results,
please contact me via email.
LS Approach Survey Results © Christian Kählig 2

3. Agenda
Executive Summary
Complete Results
LS Approach Survey Results © Christian Kählig 3

4. The Lean Startup Approach survey was one of four data collection
means used to gather data about the Lean Startup phenomenon
February 2011 March 2011
CW M T W T F S S M CW T W T F S S CW M T W T F S S CW M T W T F S S
8 21 22 23 24 25 26 27 28 9 1 2 3 4 5 6 10 7 8 9 10 11 12 13 11 14 15 16 17 18 19 20
Lean Startup Approach survey (web-based)
Lean Startup Approach survey (web-based)
n=78
(1)
LSM-BO LSM post-event survey (web-based)
LSM-BO LSM post-event survey (web-based)
n=14
(2)
Semi-structured interviews (via skype)
Semi-structured interviews (via skype)
Friday Saturday Sunday
• Lean Startup briefing • 2nd checkpoint • Team work
• Participant Pitches • Challenges & • Final presentation
n=8
(3)
• Team forming new tactics • Announcement of
• 1st checkpoint • 3rd checkpoint
winners
Note: LSM-BO = Lean Startup Machine Boston; (1) 30 out of 78 responses were
completely filled in; (2) 11 out of 14 responses were filled in by LSM Boston
event participants; (3) 7 LSM-BO event participants, 1 LSM-BO event mentor;
LS Approach Survey Results © Christian Kählig 4

5. Briefing
The Lean Startup Approach survey was conducted between February 24th and March
16th 2011.
A web-based SaaS was used to collect quantitative and qualitative data from former,
current and prospective LSM participants interested in the Lean Startup Approach.
78 people participated in the survey – 40% filled in the questionnaire completely.
The questionnaire was comprised of 10 mandatory and 18 optional questions and
covered three main areas of interest:
• Lean Startup methodology knowledge, experience and resources used
• Startup development progress determination and metrics
• Business model documentation
The survey was announced in the LS circle group, the LSM Alumni group, the LSM
Boston facebook fanpage and tweeted to current and former LSM event participants.
LS Approach Survey Results © Christian Kählig 5

6. Executive Summary – Major Findings
• The basic profile of all survey participants revealed 18 LSM Boston attendees and overall middle-aged, well-
educated US citizens
Participants’
• The majority declared extensive professional experience and considerable scalable startup experience
profile • While many noted theoretical knowledge, only a few declared extensive practical experience with the Lean
Startup methodology
• “Question your assumptions” and “Talk to your customers” were the two most important LS principles in the
eyes of the participants
• At the same time, “Question your assumptions” was seen as most difficult to implement together with “Get out
Lean Startup of the building”
principles • Comparing the views of advanced and less LS experienced participants yielded notable differences only
regarding iteration speed and pivoting
• “Validate learning” appeared to be especially difficult to implement for less experienced LS practitioners
• A notable third claimed to not document any business model related assumptions despite the stated
Assumptions doc importance of Question your assumption
Startup • Among the huge variety of elaborated startup development progress metrics, LS principles played a certain role
but did not dominate
development • Among the different customer interest measures, no clearly dominating theme could be identified
progress and • Customer related metrics dominated the most important metrics of strategic decision making about the future
metrics direction of the startup
• Blogs and tweets of Eric Ries and Steve Blank were the two most frequently noted sources of expert
Helpful information
resources • Expert advice and local meetups were indicated as most helpful resources for startup and opportunity
development
LS Approach Survey Results © Christian Kählig 6

7. Executive Summary – Discussion
• The principles “Question your assumptions”, “Getting out of the building” and “Talk to customers” were
recognized as very important, but at the same time also stated as most difficult to implement. Furthermore, no
additional principles were mentioned although – for example – Eric Ries mentions “Build-Measure-Learn” as a
core principle on his upcoming book landing page. Given the fact that nearly half of the answers (18) came from
LSM event participants, the choice of the more customer development related principles in the “most
Lean Startup important” part might be related to the nature of the LSM event with its increased emphasis on the customer
principles development part of the LS approach potentially causing some kind of bias.
• None of the chosen top three principles are actually core LS principles, but rather subordinated tenets or
principles belonging to the customer development (CD) model. A likely reason for that result could be that the
demarcation between the LS Approach principles as such and the subordinated CD related tenets (or principles)
is not strictly differentiated by the practitioners.
• A notable third of the participants stated to not document any assumptions with regard to the proposed
business model areas at all. Given the stated importance of related principles like “Question your assumptions”
and “Pivot as necessary” where one changes direction but stays grounded in what was learned before actually
gave reasons to expect a certain, pragmatic degree of model related assumption documentation and ultimately
Assumptions doc a higher number of people stating that behavior. A possible explanation that the notion of documentation
might have mistakenly evoked the business plan style of documentation – which was not the intention of
course – has not been confirmed according to the email feedback of one case interviewee.
• The fact that “value propositions” and “customer segments” were among the “most documented” assumptions
was not a real surprise and is expected to be in line with other research findings.
• Although a variety of approaches, metrics and tools were mentioned when it comes to startup development
Startup progress measures and strategic decision making, yet the customer and its attributes appeared to take a
central role given the different kind of noted elements. Additionally, not only quantitative, but also qualitative
development and even hybrid measures were declared reflecting the variety of approaches and potentially the people’s
progress and different mindsets (number-focused vs. people-focused). Only a few people directly referred to LS principles
metrics and concepts like validated learning and pivoting which could be related to a lack of awareness, experience or
acceptance as a valid means of progress measure.
LS Approach Survey Results © Christian Kählig 7

8. Agenda
Executive Summary
Complete Results
LS Approach Survey Results © Christian Kählig 8

9. Profile of the Survey participants
LS Approach Survey Results © Christian Kählig 9

10. The basic profile of all survey participants revealed 18 LSM Boston
attendees and overall middle-aged, well-educated US citizens
Age
n=28, one answ er only
11 11
4
2
16-24 25-34 35-49 50-64
Note. * Mandatory question. LSM Boston participants are highlighted in orange.
Highest degree
n=28, one answ er only
12
10
3
2
1
High school Professional degree Bachelor's degree Master's degree Doctorate degree
graduate or (e.g. MD, DDS, DVM, (e.g. BA, AB, BS) (e.g. MA, MS, MEng, (e.g. PhD, EdD)
equivalent (e.g. LLB, JD) MEd, MSW, MBA)
GED)
LS Approach Survey Results © Christian Kählig 10

11. The majority declared extensive professional experience and
considerable scalable startup experience
Professional experience
n=28, one answ er only # of scalabe startups (among first 5, > 6 months)
n=29, one answ er only
17
10 10
7
6
3
2 2
0
1-2 years 3-5 years 6-10 years 11+ years
0 1 2-3 4-5 6+
LS Approach Survey Results © Christian Kählig 11

12. While many noted theoretical knowledge, only a few declared
extensive practical experience with the Lean Startup methodology
Breadth of practical Lean Startup knowledge * Depth of practical LS knowledge *
n=41, one answ er only n=41, one answ er only
16 20
13
8
9
7
4 5
Not used in a project / Uesd in 2-4 projects / Used in 1 project / startup Used in more than 5 No integration into startup Little to some integration Advanced integration w ith Strong integration w ith
startup yet startups projects / startups activities w ith startup activities startup activities almost all startup activities
Note. * Mandatory question. Note. * Mandatory question.
Degree of theoretical Lean Startup knowledge *
n=41, one answ er only
15
14
6 6
None Read/know some of the Read/know all of the pre- Consider myself a Lean
pre-event information event information Startup expert
LS Approach Survey Results © Christian Kählig Note. * Mandatory question. 12

13. Most important and most difficult Lean Startup principles
LS Approach Survey Results © Christian Kählig 13

14. Question your assumptions and Talk to customers were the two most
important LS principles in the eyes of the participants
Key Lean Startup principles in your view
n=40, m ultiple answ ers possible
Question your assumptions 73%
Talk to customers 68%
Get out of the building 60%
Validate learning 58%
Build minimum viable products 58%
Iterate rapidly 53%
Pivot as necessary 48%
Avoid premature scaling 20%
Another principle not listed 0%
LS Approach Survey Results © Christian Kählig 14

15. At the same time, Question your assumptions was seen as most difficult
to implement together with Get out of the building
Most difficult Lean Startup principles to implement
n=38, m ultiple answ ers possible
Get out of the building 39%
Question your assumptions 37%
Validate learning 29%
Build minimum viable products 26%
Talk to customers 26%
Pivot as necessary 21%
Avoid premature scaling 13%
Iterate rapidly 8%
Another principle not listed 0%
LS Approach Survey Results © Christian Kählig 15

16. Comparing the views of advanced and less LS experienced participants
yielded notable differences only regarding iteration speed and pivoting
Lean Startup principles: most important (advanced/strong vs. little experience)
n=14 (little experience), n=9 (advanced/strong experience)
Most important - group 'advanced/strong
experience'
Most important - group 'little experience'
Question your assumptions 78%
71%
Talk to customers 78%
64%
Build minimum viable products 67%
50%
Iterate rapidly 67%
43%
Pivot as necessary 67%
36%
Get out of the building 56%
64%
Validate learning 44%
64%
Avoid premature scaling 11%
14%
LS Approach Survey Results © Christian Kählig 16

17. Validate learning appeared to be especially difficult to implement for
less experienced LS practitioners
Lean Startup principles: most difficult (advanced/strong vs. little experience)
n=14 (little experience), n=9 (advanced/strong experience)
Most difficult - group 'advanced/strong experience'
Most difficult - group 'little experience'
Question your assumptions 56%
36%
Talk to customers 56%
36%
Build minimum viable products 22%
43%
Iterate rapidly 22%
14%
Pivot as necessary 22%
21%
Get out of the building 22%
7%
Validate learning 11%
43%
Avoid premature scaling 0%
7%
LS Approach Survey Results © Christian Kählig 17

18. Documentation of Business Model assumptions
LS Approach Survey Results © Christian Kählig 18

19. A notable third claimed to not document any business model related
assumptions despite the stated importance of Question your assumption
Documentation of business model related assumptions
n=34, multiple answ ers possible
Value propositions 53%
Customer segments 53%
Revenue streams 38%
Cost structure 38%
Key activities 35%
I do not document them 32%
Channels 26%
Key resources 24%
Customer relationships 24%
Key partners 21%
LS Approach Survey Results © Christian Kählig 19

20. Startup development progress, customer interest and metrics
LS Approach Survey Results © Christian Kählig 20

21. Among the huge variety of elaborated startup development progress
metrics, LS principles played a certain role but did not dominate
Determination or measures of startup development progress (n=29, sorted alphabetically)
Categories Elements
Classic project metrics roadmap milestones, assignments, working releases
#, feedback (+/-), feedback conversion rate, health, value added, # of talks with important results,
Customer metrics
consider product must-have
Lean startup principles validated learning, pivots, # of interviews before P/S fit, first MVP
Products/Services metrics level of functionality, transactions per user
Revenue metrics general, per customer, # of paying customers
Self-assessment actionable steps, degree of focus, qualitiative
Time/Cost metrics time and cost to deploy first minimum viable solution, time and cost for subsequent value-adds
Tools dashboard, Pivotal Tracker
Web(-related) metrics AARRR*, unique visitors, users, activity, bounces, unbounces, social media mentions
Note. * AARRR is a term coined by Dave McClure and represents: acquisition, activation, retention, referral, revenue
Note. Inductive category creation (Flick, 2009) was used to compile the categories and elements (sub-categories) above based on the
participants‘ answers to the respective open-ended question.
LS Approach Survey Results © Christian Kählig 21

22. Among the different customer interest measures, no clearly dominating
theme could be identified
Determination or measures of customer interest (n=26, sorted alphabetically)
Categories Elements
AARRR acquisition, activation, retention, referral, revenue
Commitment willingness-to-use, -pay, -refer
Conversion funnel, cost
Interviews qualitative, face-to-face, phone, reaction, follow-ups, feedback quality, positive feedback, surveys
Landing page traffic, activitiy, signups
Problem significant pain solved, degree of disappointment in case product/service would disappear
Note. Inductive category creation (Flick, 2009) was used to compile the categories and elements (sub-categories) above based on the
participants‘ answers to the respective open-ended question.
LS Approach Survey Results © Christian Kählig 22

23. Customer related metrics dominated the most important metrics of
strategic decision making about the future direction of the startup
Metrics for important strategic decisions about startup direction (n=27, sorted alphabetically)
Categories Elements
feedback, interest, passion, problem assumptions validation, scalable acquisition strategy,
Customer willingness-to-pay, new per segment, actions vs. expectations, value recognition, types
understanding, service cost, new adoption rate
Learning pace
market size, marketplace awareness, general funnel thinking, product/market fit, main area of
Marketing and Sales
growth
Others business specific, assumed landscape in 6 months, meeting with stakeholders
Product and Solution time and cost for first minimum viable solution, problem/solution fit
Revenue/Profit general, early profit
Strategic positioning change relative to customers and competitors
User #, activity, signups, # of signups vs. users, conversion
Note. Inductive category creation (Flick, 2009) was used to compile the categories and elements (sub-categories) above based on the
participants‘ answers to the respective open-ended question.
LS Approach Survey Results © Christian Kählig 23

24. Helpful Resources
LS Approach Survey Results © Christian Kählig 24

25. Blogs and tweets of Eric Ries and Steve Blank were the two most
frequently noted sources of expert information
Expert blogs or tweets read by practitioners
n=32, m ultiple answ ers possible
Eric Ries 91%
Steve Blank 88%
Dave McClure 66%
Ash Maurya 53%
Patrick Vlaskovits 50%
Brant Cooper 44%
Sean Ellis 38%
Alexander Osterwalder 28%
Andrew Chen 6%
Cindy Alvarez 6%
Tor Gronsund 3%
Nathan Furr 3%
Tom Eisenmann 3%
Trevor Owens 3%
Jason Cohen 3%
Neil Patel 3%
Hiten Shah 3%
LS Approach Survey Results © Christian Kählig 25

26. Expert advice and local meetups were indicated as most helpful
resources for startup and opportunity development
Most helpful resources so far (n=23, sorted alphabetically)
Categories Elements
Expert advice blogs (14), books (10), lsc group (4), videos (2), tweets (1)
Local meetups local lean startup meetups (3)
Tools business model canvas (1)
Note. Number of mentions in parantheses. Elements ordered from most to least frequent mentions.
Note. Inductive category creation (Flick, 2009) was used to compile the categories and elements (sub-categories) above based on the
participants‘ answers to the respective open-ended question.
LS Approach Survey Results © Christian Kählig 26

27. Christian Kählig
f: facebook.com/ideasenabled
w: ideasenabled.com
l: de.linkedin.com/in/ckaehlig
… for your contribution to this research endeavor!
LS Approach Survey Results © Christian Kählig 27