In this class, we will explore how, given a target market and chosen user persona, we can come up with a solution that solves these problems and meets the needs and wants of the customer. We will discuss how to come up with ways to develop high level product concepts that can be tested in the field. We will cover hypothesis testing techniques, including classical quantitative techniques like usability benchmarks as well as modern techniques such as the use of landing pages to test product interest and purchase intent.

1. MIT Entrepreneurship and
Maker Skills Integrator
What can you do for your customer? |
Hypothesis testing
January 10, 2017
Elaine Chen

2. What we are going to cover today
2
1. DE Theme 2: What can you do for your customer?
2. How to do hypothesis testing with MV(B)Ps (in particular:
How to test purchase intent)

3. DE Theme 2: What can you do for your
customer?

4. “What can you do for your customer?”
4
1 operator,
2 shifts
1 Baxter, 2
shifts
Savings / Gain
Monthly
Cost
$8,680 $52 $8,628
Yearly cost $104,166 $625 $103,536
3y cost $312,498 $1,875 $310,608
6
7
8
10
11

5. 5
Step 6: Full Cycle Use Case
Baxter training (2011 – 10th or 15th iteration)

6. Final product: Baxter Training Use Case
6
1. The process engineer picks a site for
Baxter and scopes out a line loading task.
He maps it all out in his head – which
hand picks what up and in what order
2. The process engineer starts training Baxter by
grapping its right wrist cuff, guiding the arm to the
location, then teaching it to find and grasp a part from the
pick up location. He then moves the arm to the place
location and teaches Baxter to let go.
3. Rinse
and repeat
with the
other arm.
4. Fine tune sequencing /
counting / other details via
on-board controls and
5. Step back and let Baxter work,
fine tune if needed. If not: Turn over
to operator for care and feeding.

7. In-class exercise: #1 Use Case
Enumerate the top 3-5 use cases for your venture.
Pick the #1 use case and write down the rationale for why this is #1.
Define your #1 full cycle use case for your primary end user persona
when using your product or service.
Do this from the perspective of the end user. Be as detailed as
possible about what the customer has to do to achieve their goals
7

8. Sharing!
8

9. Step 7: High level product spec
9

10. High level product spec, take 2
10

11. Before and after
11
Feature focus, pushes the technology Benefits focus, pushes UVP/QVP

12. Another example: Zeo
Zeo MobileZeo 2.0 Bedside Display
Universal Bluetooth Headband
Zeo Sleep
Hub
Environmental
Sensor
Accessory
Secondary
Headband
Other
Sensor
Accessories
The Universal Headband measures sleep
quality continuously by recording and
analyzing EEG data through the night.
The bedside display can show the user
their sleep graph from the previous
night. It also uploads the data via WiFi.
This device can also connect to other
Bluetooth sensors in the future.
The Zeo mobile application can
show the user their sleep graph. It
also uploads data via a WiFi or
Cellular connection.
The Zeo Sleep Hub is a data aggregation,
visualization and analytics engine in the cloud
In the morning, the data is sent via
Bluetooth to either a bedside
display or a mobile device.

13. Define it before you build it
• Before you invest valuable resources, test first
• 1. If whole team agrees on what you should build
• 2. That you are building what the customer wants
• Do this by making a “brochure” or landing page
• Allows for spiraling innovation. Build, measure, learn.

14. Spiraling Innovation
14

15. VALUE PROPOSITIONStep 8: Quantified Value Proposition Baxter
@ Valin
Task: transferring paint cylinders from one
conveyor to another
How it’s done today: 1 operator per shift, 2
shifts, 100% loading
How it’s done with Baxter: 1 Baxter working
continuously over 2 shifts; operators doing
value added tasks elsewhere in the plant
Robot price: $32,000 fully loaded
Robot operating cost: $52/m (electricity cost)
Savings analysis: 4 MONTHS ROI
1 operator per shift x
2 shifts
1 Baxter, 2 shifts Savings/Gain
Monthly Cost $8,680 (@$25/h) $52
(Electricity cost)
$8,628
Yearly cost $104,166 $625 $103,536
3y cost $312,498 $1,875 $310,608

16. Step 10: Define your core
16
Hardware core Software / systems core

17. What is the core for each of these companies?
17
Subscriber list
Online fulfilment etc etc
Customer Experience
Hardcore technology
Search etc etcOperational excellence

18. Step 11: Competitive 2x2 Example:
Industrial automation
Changeover time to accommodate new jobs
Slow (months/years)Fast (days)
Expensive InexpensiveTotal cost of ownership

19. Homework: Competitive 2x2
Come up with the top 2 priorities that are the most important to
your customers (either economic buyer or user – pick one if
they are different). These are the 2 axes of your 2x2.
Enumerate the top 5-10 competitors that solve a similar
problem you are solving. Note: “Do nothing” is always a
competing option.
Create a 2x2 in which your offering is up and to the right and
everyone is somewhere else – and be ready to talk to what
makes you special.
19

20. Sharing!
20

21. Product Development Must be a Rapid &
Continuous Process
21
© Ash Maurya, “Running Lean”, www.ashmaurya.com

22. “What can you do for your customer?” - reprise
22
1 operator,
2 shifts
1 Baxter, 2
shifts
Savings / Gain
Monthly
Cost
$8,680 $52 $8,628
Yearly cost $104,166 $625 $103,536
3y cost $312,498 $1,875 $310,608
6
7
8
10
11

23. Hypothesis testing
“A good hypothesis
needs to be written
down.”
– Ben Yoskovitz, Co-Author, “Lean Analytics”

24. On hypotheses
• Basic structure:
• I believe [target market] will [do this action /
use this solution] for [this reason].
24

25. Example hypotheses
25
I believe the Frustrated Sleeper will wear a
headband every night to understand what really
happened during their night of poor sleep.
I believe users of this iOS app will quickly
learn how to use our gesture UI to dig
through layers of data on a single screen.
I believe plant managers will buy Baxter without
a proof-of-concept because it is so cheap.

26. Hypothesis testing with MV(B)P
• Hypothesis testing using MVP tests the product
• Hypothesis testing using MVBP tests market viability
• A proper experiment looks like this:
1. Write down your hypotheses and define your MVP
2. Design and run the experiment
3. (Optional) Set a measurable threshold for the action, and use the results
to decide whether you keep going, tweak the approach, or pivot. (This is
very good practice!)
26

27. Which hypothesis should you test first?
27
• Riskiest
• Longest lead time
• Costliest if invalidated
… etc
Pick the first hypothesis you want to test – and define a
corresponding MVP to test it with

28. Case Study: New, Affordable Lending
Product for consumers in Mexico City
• There is demand for consumer loans between USD$100 - $500 in Mexico
• Customers most likely to use this product are:
• Age: 35-45
• Locality: Urban/semi urban
• Education / income source: High school; formal salaried / informal entrepreneur
• Income range: USD $7,200 – $9,600
• Most customers need the funds for:
• School fees
• Home improvement
• Debt consolidation/refinancing
• Auto loan
• Customers care about interest rates
• Customers will be willing to download our app in order to obtain a loan
• Customers would trust an unknown company to provide them with financial services
• Customers do not have other attractive options for obtaining these funds
• Customers need a convenient method for cash in/out
28

29. Case Study: New, Affordable Lending
Product for consumers in Mexico City
• There is demand for consumer loans between USD$100 - $500 in Mexico
• Customers most likely to use this product are:
• Age: 35-45
• Locality: Urban/semi urban
• Education / income source: High school; formal salaried / informal entrepreneur
• Income range: USD $7,200 – $9,600
• Most customers need the funds for:
• School fees
• Home improvement
• Debt consolidation/refinancing
• Auto loan
• Customers care about interest rates
• Customers will be willing to download our app in order to obtain a loan
• Customers would trust an unknown company to provide them with financial services
• Customers do not have other attractive options for obtaining these funds
• Customers need a convenient method for cash in/out
29

30. Workshop activity: Hypotheses
Define the top 3-5 riskiest hypotheses that, if invalidated, could
cause a pivot for your venture.
For each hypothesis, use this structure (or at least start with “I
believe”):
I believe [target market] will [do this action / use this solution] for
[this reason].
Pick 1 hypothesis to be tested first
30

31. Sharing!
31

32. Workshop 3: Digital experimentation

33. A typical digital experiment
33
Design landing
page(s) with a call
to action
State the
hypothesis
Design an
experiment
Send out landing
page to suspects
Wait for results to
come in
Analyze sales
funnel metrics
Persevere or pivot?

34. About that landing page…
34
Design landing
page(s) with a call
to action
State the
hypothesis
Design an
experiment
Send out landing
page to suspects
Wait for results to
come in
Analyze sales
funnel metrics
Persevere or pivot?

35. Things you can test with landing pages
• Product interest
• Purchase intent
• Product positioning
• Product concept messaging
• Visual design
• ... etc
35

36. The story of Accumark 3D
36

37. 4h to find out if there is interest
37
Opens Email
Clicks thru to
pageClicks “Sign
up”Provide
s email
30%
It’s a GO!

38. Example landing page – cheap “currency”
38

39. Example landing page - real ”currency”
39

40. Experimental platform: http://unbounce.com
40

41. In your copious free time
• Goal: Design an experiment to gauge interest in a
new offering at the HK Node – a speaker series for
hardware entrepreneurs at the Hong Kong node
• Exercise:
• Pick a target persona and explain how you will reach
them
• Design the experiment including success criteria
• Sketch the landing page
41

42. Sharing!
42

43. Pro tips
• Treat the landing page like an ad! It is a
marketing creative exercise.
• Collect “currency” in the call to action
• Email is meaningful
• Credit card is even more meaningful
• Pre-payment is the most meaningful
• Graph and interpret funnel metrics
43

44. Supplemental Readings
• “Our dangerous obsession with the MVP” – Bill Aulet,
Techcrunch
• “Defining and Building the MVP” – Elaine Chen
(Slideshare)
• “Product definition templates” – Elaine Chen (Slideshare)
• “Validating Hypotheses with Landing Page Tests” - Elaine
Chen (PDF)
44

45. End
Questions?
45

46. Backup Slides

47. Use case example: Playrific Mobile
47
1 2 3
4 5 6
Product idea:
A safe iPhone app
that provides
curated, high quality
edutainment content
to keep pre-reading
children occupied
while the family is on
the go

48. High Level Product Spec Counter Example:
The Airborne Wind Turbine
The Airborne Wind Turbine integrates proven aerospace and wind turbine technology. The AWT
lifting platform is adapted from tethered aerostats, which have reliably lifted heavy
communications and monitoring equipment high into the air for decades. The AWT integrates
four main components:
• Lifting Shroud - A proprietary helium-filled shroud made from high performance, industrial
fabrics that lifts the turbine up and stabilizes it in the air.
• Wind Turbine - A lightweight conventional three-blade, horizontal axis wind turbine fixed
within the shell.
• Conductive Tethers – The lightweight, high strength tethers hold the turbine in place in all
weather conditions and transmit power to the ground.
• Base Station - The portable base station is rapidly deployed from a shipping container and
includes an autonomous control system and power conditioning equipment.

49. 49
• Tether holds AWT in place at optimal height and
delivers electricity down to onshore base-station or
offshore buoy, and onto electrical grid or end user
• Shroud uses buoyant and aerodynamic lift to elevate
lightweight wind turbine to harness strong winds 200-
600 meters in the air
Lifting shroud & Wind
Turbine
Conductive Tether &
Base-Station
200-600meters
Computer
Model of the
patent-pending
Airborne Wind
Turbine (AWT)
The Airborne Wind Turbine (AWT)

50. Quantified Value Proposition Example: Zeo
(Bad example)
50

51. VALUE PROPOSITION
1. SIX times longer duration
2. FIVE times higher effectiveness
3. At least THREE times lower quantity applied.
Quantified Value Proposition
Example: Smart Skin Care
Long-lasting protection for your skin

52. Quantified Value Proposition Example: 3D
Samples for Apparel
Physical process: 5 concept samples, 4 production samples, 18 weeks
1st patterns Make Sample
Tweak
patterns
Make
Sample
Tweak
patterns
Tweak
patterns
Tweak
patterns
Make
Sample
Make
Sample
Make
Sample
Production
patterns
Make Proto
sample
Tweak
patterns
Make Proto
Sample
Tweak
patterns
Make TOP
sample
Make
photoshoot
sample
Start mass
production
Tweak
patterns
Make Proto
Sample
1st patterns
Make
Sample
Tweak
patterns
Make
Sample
Tweak
patterns
Make
Sample
Production
patterns
Make Proto
sample
Tweak
patterns
Make TOP
sample
Make
photoshoot
sample
Start mass
production
3D review;
iterations
3D review;
iterations
With 3D Samples: 3 concept samples, 2 production samples, 10-12 weeks
SAVINGS:
6-8 weeks elapsed time
$1k x 4 samples x 500 styles =
$2 M savings per season

53. Quantified Value Proposition Example: Altaeros
Energy
53
“over twice the energy of similarly
rated tower-mounted wind turbines”
“reduces the installation and transport
cost by up to 90%”
“can be installed and producing power in
under 24 hours”

54. Competitive 2x2 Example
Ease of migration
LowHigh
Low High
Enterprise readiness: Comprehensive SCM management tools + ALM coverage
Awesome B2B Enterprise
Software Code
Management product
from Company X

55. Example MVPs
55
Fake, non-functional
headband
Ugly interactive prototype
Product brochure plus a video
clip

56. Case study: iOS App MVP Test
56
Hypothesis: “I believe users will find the
gesture UX delightful.”
MVP: Functional interactive prototype
Experiment: Usability benchmark. Users
were given tasks to complete that required
them figuring out the gesture UX on their
own.
Duration: 30 min UX test + 15 min debrief
Threshold: 50% of users will complete the
task.Result: 0% figured it out.
We killed the feature.

57. Put the “B” in the “MVBP”
57
You can run surveys until the cows come home, but you
won’t know for sure until $ changes hands.

58. Testing willingness to pay before there’s anything to
sell: The concept of “currency”
58
Short of taking $$$ from your potential customers for your nonexistent
vaporware, what are some asks you can pose to collect surrogate currency that
is a predictor of future willingness to pay?
• Yes to scheduling a meeting to discuss
• Actually getting a scheduled meeting on the calendar
• Actually meeting you
• Giving you the email of someone else you should meet
• E-introducing you to someone else you should meet
• Providing an email
• Providing Credit Card for a pre-order (no charge)
• Providing Credit Card for a pre-order (charged, rain check provided)
• … etc

59. Example 1: Landing Page Test
59
Opens Email
Clicks thru to
pageClicks “Sign
up”Provide
s email
30%
It’s a GO!

60. Example 2: Kickstarter
60

61. Example 3: Preorder on own site
61

62. Example 4: Paid concierge beta
62
Hypothesis: “User will pay for this SaaS
visualization service”.
MVP: Fully faked via emails and Photoshop
Experiment: 10 customer, paid beta
Duration: 3 months
Currency: $199/month, prepaid
Threshold: Able to close more than 5 for beta
They closed 10. It’s a GO!
Extreme example of “get paid from day 1” without a
line of code