O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
CONFIDENTIAL | 1
CONFIDENTIAL | 2
Featured Project:
The Wroclaw West Bypass, Poland
Trusted by the world’s largest projects
Pivots for real...
CONFIDENTIAL | 3
Obligatory Who am I
CONFIDENTIAL | 4
BidContender
• Online Tender Management platform that has connected builders and subbies
on $48 billion i...
CONFIDENTIAL | 5
Stories -> Hypothesis
120 Points =
done
http://www.agileproductdesign.com/
CONFIDENTIAL | 6
How to know what’s of value?
• Experiments, validated learning, testing things
Read it, loved it, but… wh...
CONFIDENTIAL | 7
Torres format hypothesis
http://www.producttalk.org/2014/11/the-5-components-of-a-good-hypothesis/
CONFIDENTIAL | 8
Hypothesis Wall
CONFIDENTIAL | 9
CONFIDENTIAL | 10
CONFIDENTIAL | 11
Data Measures Outcomes
CONFIDENTIAL | 12
Share your data using online apps, not IT, and not Excel
Geckoboard
DataHero
CONFIDENTIAL | 13
Then build until the number moves…
4 Weeks
CONFIDENTIAL | 14
or doesn’t.
2 weeks
CONFIDENTIAL | 15
Data can be sourced from anything…
• Don’t feel you need to build amazing infrastructure to get it
CONFIDENTIAL | 16
Fake it `til you make it.
CONFIDENTIAL | 17
TV’s are essential
CONFIDENTIAL | 18
The motivation is palpable
CONFIDENTIAL | 19
The journey from spec to outcome
CONFIDENTIAL | 20
An example
Hypothesis
2 X Transactions
FeaturesOutcomes
$
Say, 20/ month 40/ month &
growing
Many
iterat...
CONFIDENTIAL | 21
Waterfront City, Australia | US $788M Value
Goals drive hypotheses
CONFIDENTIAL | 22
Goal Hypothesis* Measure
* Many of these,
in a hierarchy
CONFIDENTIAL | 23
Discover the drivers for your goals (Impact Mapping)
The Goal
Who needs to
change
Ways we can
drive chan...
CONFIDENTIAL | 24
The ‘guesses’ down the tree can be stated as
hypothesis
.
We believe that if we
implement semi
automated...
CONFIDENTIAL | 25
Goals are the outcome at the end of a funnel
Step 1
Step 2
Step
3
% that get to
this step is the
goal
10...
CONFIDENTIAL | 26
Funnel maps steps through a product workflow
Step 1
Step 2
Step 3
Step 1
Step 2
Step
3
CONFIDENTIAL | 27
We took a flow from our app and created
hypothesis for change
STEPS
Registered
Profile Updated
Subscribe...
CONFIDENTIAL | 28
Hypothesis for changes to a step
STEPS
Registered
Profile Updated
Subscribed
Subbie connected
to builder...
CONFIDENTIAL | 29
Hypothesis for changes to a step
STEPS
Registered
Profile Updated
Subscribed
Subbie connected
to builder...
CONFIDENTIAL | 30
Inference Tree
CONFIDENTIAL | 31
Inference Tree
CONFIDENTIAL | 32
Increasing Activation to 90% will
increase retention by 20% for pro
subscribers within a quarter
Ensurin...
CONFIDENTIAL | 33
And then build it…
Adding a list of builders in their
state will increase connections to
builders for su...
CONFIDENTIAL | 34
There’s the measure:
CONFIDENTIAL | 35
Indira Gandhi International Airport, Terminal 3, India | US $2B Value
Data informs hypotheses.
Passing H...
CONFIDENTIAL | 36
Further Reading
http://www.businessmodelgeneration.com/book
http://practicetrumpstheory.com/http://www.a...
CONFIDENTIAL | 37
The Center to Grand Forks (CGF) Project, U.S.A. | US $350M Value
We’re Hiring!
http://Aconex.com/careers
CONFIDENTIAL | 38
Seattle Tunnel State Route 99, U.S.A | US $1.4B Value
Thank You
CONFIDENTIAL | 39
Preparation (for 2nd Session – if you’re coming along!)
• Bring your laptop (or use one of the computers...
CONFIDENTIAL | 40
CONFIDENTIAL | 41
Dubai International Airport, U.A.E | US $4.5B Value
Session 2: Using
Geckoboard and
DataHero to measure
...
CONFIDENTIAL | 42
Yas Island, U.A.E. | US $40B Value
How do I turn my backlog
of stories into
hypotheses?
CONFIDENTIAL | 43
Some techniques
• Extract the assumptions/intention from the story
• As a <who>
• I want to <the change>...
CONFIDENTIAL | 44
But Why?
• The 5 why’s exercise on a stories ass:
– Q: Why do I want to get semi automated invites?
– A:...
CONFIDENTIAL | 45
Check the (HIPPO’s) bullshit
Bullshit! Will semi
automated REALLY
drive more friends?
• The 5 why’s exer...
CONFIDENTIAL | 46
Let’s try it…. Volunteer?
• Let’s take someone’s backlog and have a go at converting it to an inference
...
CONFIDENTIAL | 47
Measures are easy in geckoboard
• Pick any piece of data you need for your inference tree
• Sign in to G...
CONFIDENTIAL | 48
Dubai International Airport, U.A.E | US $4.5B Value
Demo: Geckoboard
CONFIDENTIAL | 49
Dubai International Airport, U.A.E | US $4.5B Value
Demo: DataHero
CONFIDENTIAL | 50
Dubai International Airport, U.A.E | US $4.5B Value
OK – your turn
CONFIDENTIAL | 51
Get going!
Your turn – in groups of 4:
– Take your backlog, and find the real goal
– Find the assumption...
CONFIDENTIAL | 52
Objections to Dashboards
• Data Objections:
– Don’t have access. (you don’t have permissions to that dat...
CONFIDENTIAL | 53
Seattle Tunnel State Route 99, U.S.A | US $1.4B Value
Visualisation
CONFIDENTIAL | 54
Static Numbers tell us NOTHING!
That’s nice. Are
we better or
worse than last
month/ pre
change?
CONFIDENTIAL | 55
Stacked time series is better
Oh, up and to
the right….
But why did it
slow?
CONFIDENTIAL | 56
Sparklines show some history – puts the number
in context!
CONFIDENTIAL | 57
Set a goal – so motivating
This goal is
probably too far!
Just to 5 would
be great right
now!
CONFIDENTIAL | 58
The % change – when you want to see increments
CONFIDENTIAL | 59
Seattle Tunnel State Route 99, U.S.A | US $1.4B Value
Never forget you are
telling a story
CONFIDENTIAL | 60
Pro Activation Funnel has improved from our work….
• We’ve moved from 25% (28-May-2015) to 41% (3-Jun-20...
CONFIDENTIAL | 61
And look at starter!
• This has been sitting
on 7% for some
weeks (haven’t
captured the
progression).
• ...
CONFIDENTIAL | 62
Still a problem, but not enough to finish?
• Drop off on connection
requests
– so we’re doing
reminder e...
CONFIDENTIAL | 63
Seattle Tunnel State Route 99, U.S.A | US $1.4B Value
Thanks – questions?
CONFIDENTIAL | 64
The Center to Grand Forks (CGF) Project, U.S.A. | US $350M Value
We’re Hiring!
http://Aconex.com/careers
Próximos SlideShares
Carregando em…5
×

Pivots for real for LASTConf sept 2015

1.284 visualizações

Publicada em

Discusses techniques I've used to do Lean Startup style pivots in an agile team and web application. Presented at LAST Conference 2015. We use impact mapping, geckoboard, datahero to show how having a hypothsis backed by data has helped us make decisions to improve bidcontender.com.

Publicada em: Software
  • Login to see the comments

Pivots for real for LASTConf sept 2015

  1. 1. CONFIDENTIAL | 1
  2. 2. CONFIDENTIAL | 2 Featured Project: The Wroclaw West Bypass, Poland Trusted by the world’s largest projects Pivots for real: how we've hacked up some data to know we've hit the mark Mark Smith @markjdsmith
  3. 3. CONFIDENTIAL | 3 Obligatory Who am I
  4. 4. CONFIDENTIAL | 4 BidContender • Online Tender Management platform that has connected builders and subbies on $48 billion in projects this year
  5. 5. CONFIDENTIAL | 5 Stories -> Hypothesis 120 Points = done http://www.agileproductdesign.com/
  6. 6. CONFIDENTIAL | 6 How to know what’s of value? • Experiments, validated learning, testing things Read it, loved it, but… when it came to it. How?
  7. 7. CONFIDENTIAL | 7 Torres format hypothesis http://www.producttalk.org/2014/11/the-5-components-of-a-good-hypothesis/
  8. 8. CONFIDENTIAL | 8 Hypothesis Wall
  9. 9. CONFIDENTIAL | 9
  10. 10. CONFIDENTIAL | 10
  11. 11. CONFIDENTIAL | 11 Data Measures Outcomes
  12. 12. CONFIDENTIAL | 12 Share your data using online apps, not IT, and not Excel Geckoboard DataHero
  13. 13. CONFIDENTIAL | 13 Then build until the number moves… 4 Weeks
  14. 14. CONFIDENTIAL | 14 or doesn’t. 2 weeks
  15. 15. CONFIDENTIAL | 15 Data can be sourced from anything… • Don’t feel you need to build amazing infrastructure to get it
  16. 16. CONFIDENTIAL | 16 Fake it `til you make it.
  17. 17. CONFIDENTIAL | 17 TV’s are essential
  18. 18. CONFIDENTIAL | 18 The motivation is palpable
  19. 19. CONFIDENTIAL | 19 The journey from spec to outcome
  20. 20. CONFIDENTIAL | 20 An example Hypothesis 2 X Transactions FeaturesOutcomes $ Say, 20/ month 40/ month & growing Many iterations of price, duration offer, app, marcomms
  21. 21. CONFIDENTIAL | 21 Waterfront City, Australia | US $788M Value Goals drive hypotheses
  22. 22. CONFIDENTIAL | 22 Goal Hypothesis* Measure * Many of these, in a hierarchy
  23. 23. CONFIDENTIAL | 23 Discover the drivers for your goals (Impact Mapping) The Goal Who needs to change Ways we can drive change Down here – you’re just guessing (hypothesis!) http://www.impactmapping.org/
  24. 24. CONFIDENTIAL | 24 The ‘guesses’ down the tree can be stated as hypothesis . We believe that if we implement semi automated invites we will see the number of invited friends increase x% within 1 month
  25. 25. CONFIDENTIAL | 25 Goals are the outcome at the end of a funnel Step 1 Step 2 Step 3 % that get to this step is the goal 100 users 50 users 10 users (10%)
  26. 26. CONFIDENTIAL | 26 Funnel maps steps through a product workflow Step 1 Step 2 Step 3 Step 1 Step 2 Step 3
  27. 27. CONFIDENTIAL | 27 We took a flow from our app and created hypothesis for change STEPS Registered Profile Updated Subscribed Subbie connected to builder Labelled by builder Invited to quote quoted
  28. 28. CONFIDENTIAL | 28 Hypothesis for changes to a step STEPS Registered Profile Updated Subscribed Subbie connected to builder Labelled by builder Invited to quote quoted Adding a list of ‘builders in their state’ will increase connections to builders for subbies by 50% within 1 month of introduction Hypothesis
  29. 29. CONFIDENTIAL | 29 Hypothesis for changes to a step STEPS Registered Profile Updated Subscribed Subbie connected to builder Labelled by builder Invited to quote quoted Adding a list of ‘builders in their state’ will increase connections to builders for subbies by 50% within 1 month of introduction Hypothesis Search filter for builders in Network directory Send URL to builders list in email Static list (A-Z) of builders Options to solve
  30. 30. CONFIDENTIAL | 30 Inference Tree
  31. 31. CONFIDENTIAL | 31 Inference Tree
  32. 32. CONFIDENTIAL | 32 Increasing Activation to 90% will increase retention by 20% for pro subscribers within a quarter Ensuring Subbies are being added to builders networks will increase invitations to quote by 30% for subbies within 1 month of introduction Adding a list of builders in their state will increase connections to builders for subbies by 50% within 1 month of introduction If this holds Then this may hold And this will hold (invitations create activation)
  33. 33. CONFIDENTIAL | 33 And then build it… Adding a list of builders in their state will increase connections to builders for subbies by 50% within 1 month of introduction As a subbie I want to see a list of contractors in my state so that I can connect to them As a builder I want an indication that subbies are trying to connect with me so that I will go and complete the connection As a product manager I want to know how many subbies use the ‘find contractor feature’ so I can measure the outcome Etc…
  34. 34. CONFIDENTIAL | 34 There’s the measure:
  35. 35. CONFIDENTIAL | 35 Indira Gandhi International Airport, Terminal 3, India | US $2B Value Data informs hypotheses. Passing Hypotheses deliver your goals. Failing Hypotheses don’t. Find out which is which!
  36. 36. CONFIDENTIAL | 36 Further Reading http://www.businessmodelgeneration.com/book http://practicetrumpstheory.com/http://www.amazon.com/Value-Proposition-Design-Customers-Strategyzer/dp/1118968050 http://shop.oreilly.com/product/0636920033851.do
  37. 37. CONFIDENTIAL | 37 The Center to Grand Forks (CGF) Project, U.S.A. | US $350M Value We’re Hiring! http://Aconex.com/careers
  38. 38. CONFIDENTIAL | 38 Seattle Tunnel State Route 99, U.S.A | US $1.4B Value Thank You
  39. 39. CONFIDENTIAL | 39 Preparation (for 2nd Session – if you’re coming along!) • Bring your laptop (or use one of the computers in the lab) • Ensure you know your Google Analytics credentials. • Have a google apps account so you can use google sheets • Bring a list of 5 epics from your current backlog
  40. 40. CONFIDENTIAL | 40
  41. 41. CONFIDENTIAL | 41 Dubai International Airport, U.A.E | US $4.5B Value Session 2: Using Geckoboard and DataHero to measure your hypothesis
  42. 42. CONFIDENTIAL | 42 Yas Island, U.A.E. | US $40B Value How do I turn my backlog of stories into hypotheses?
  43. 43. CONFIDENTIAL | 43 Some techniques • Extract the assumptions/intention from the story • As a <who> • I want to <the change> • So that <the measure> As a player I want to have semi automated invites sent So that my list of friends increases
  44. 44. CONFIDENTIAL | 44 But Why? • The 5 why’s exercise on a stories ass: – Q: Why do I want to get semi automated invites? – A: so I increase the list of my friends – Q: why do you want to increase the list of my friends? – A: so that we get more players – Q: Why do I want more players? – A: so that we get to 1M players – Q: why 1M players – A: Cos then I get a Ferrari…. As a player I want to have semi automated invites sent So that my list of friends increases
  45. 45. CONFIDENTIAL | 45 Check the (HIPPO’s) bullshit Bullshit! Will semi automated REALLY drive more friends? • The 5 why’s exercise on a stories ass: – Q: Why do I want to get semi automated invites? – A: so I increase the list of my friends – Q: why do you want to increase the list of my friends? – A: so that we get more players – Q: Why do I want more players? – A: so that we get to 1M players – Q: why 1M players – A: Cos then I get a Ferrari…. Bullshit! Will more friends in the list really get more players? The implied hierarchy of the 5 Why’s and the bullshit detector drives a natural inference tree
  46. 46. CONFIDENTIAL | 46 Let’s try it…. Volunteer? • Let’s take someone’s backlog and have a go at converting it to an inference tree: – Volunteer – bring out your top 5 cards – Let’s do it together.
  47. 47. CONFIDENTIAL | 47 Measures are easy in geckoboard • Pick any piece of data you need for your inference tree • Sign in to Geckoboard. • Create a dashboard • Put the data there.
  48. 48. CONFIDENTIAL | 48 Dubai International Airport, U.A.E | US $4.5B Value Demo: Geckoboard
  49. 49. CONFIDENTIAL | 49 Dubai International Airport, U.A.E | US $4.5B Value Demo: DataHero
  50. 50. CONFIDENTIAL | 50 Dubai International Airport, U.A.E | US $4.5B Value OK – your turn
  51. 51. CONFIDENTIAL | 51 Get going! Your turn – in groups of 4: – Take your backlog, and find the real goal – Find the assumptions – Write down the hypothesis as a tree of inference – Work out the data you need – Connect it to your geckoboard
  52. 52. CONFIDENTIAL | 52 Objections to Dashboards • Data Objections: – Don’t have access. (you don’t have permissions to that data) – Can’t get that yet (integration problem) – It’s in an email. • Answer: – Google Sheets. • IT objection – We’re not paying till you do a business case. – OFFS – buy it on your credit card and then show the improvement, then get the corporate card on it….. • Priority Objection – We don’t have time to build measures, we need to build feature X • (if nobody uses it, if it doesn’t convert, what’s the point!) • A good product manager knows to kill features as well as make them.
  53. 53. CONFIDENTIAL | 53 Seattle Tunnel State Route 99, U.S.A | US $1.4B Value Visualisation
  54. 54. CONFIDENTIAL | 54 Static Numbers tell us NOTHING! That’s nice. Are we better or worse than last month/ pre change?
  55. 55. CONFIDENTIAL | 55 Stacked time series is better Oh, up and to the right…. But why did it slow?
  56. 56. CONFIDENTIAL | 56 Sparklines show some history – puts the number in context!
  57. 57. CONFIDENTIAL | 57 Set a goal – so motivating This goal is probably too far! Just to 5 would be great right now!
  58. 58. CONFIDENTIAL | 58 The % change – when you want to see increments
  59. 59. CONFIDENTIAL | 59 Seattle Tunnel State Route 99, U.S.A | US $1.4B Value Never forget you are telling a story
  60. 60. CONFIDENTIAL | 60 Pro Activation Funnel has improved from our work…. • We’ve moved from 25% (28-May-2015) to 41% (3-Jun-2015) to 47% (17-Jun- 2015) • We did these changes: – Introduced auto-labelling – Pending network connections show as icon on network directory 28-may-2015 3-Jun-2015 17-Jun-2015
  61. 61. CONFIDENTIAL | 61 And look at starter! • This has been sitting on 7% for some weeks (haven’t captured the progression). • Look at the numbers on it now!
  62. 62. CONFIDENTIAL | 62 Still a problem, but not enough to finish? • Drop off on connection requests – so we’re doing reminder email tonight.) • Drop off on invite to quoted – Will cover this as part of builder onboarding (they want to get coverage).
  63. 63. CONFIDENTIAL | 63 Seattle Tunnel State Route 99, U.S.A | US $1.4B Value Thanks – questions?
  64. 64. CONFIDENTIAL | 64 The Center to Grand Forks (CGF) Project, U.S.A. | US $350M Value We’re Hiring! http://Aconex.com/careers

×