SlideShare a Scribd company logo
1 of 89
Developer's Guide to
 Feedback Driven
   Development
      Marty Haught
       @mghaught
Did you get the memo?
We’re starting a new project
Do we have requirements?
  Project release party?
Nope, not like you’re thinking
Huh?
Yeah, we’ll just build out
      a few weeks
That’s all they want?!?
Oh no, it’s just to get started
So how can we plan it out?
I’ll send you the memo
How do you build a product
   when you only have a
vision and a starting place?
FbDD

 Feedback
  Driven
Development
Session roadmap

1. What is feedback driven development

2. Role in development

3. Techniques

4. Integration
Boulder Ruby




Longmont, Colorado
1. FbDD, what is it?

“A collection of techniques for
measuring progress based on
    customer interaction.”
Via lean startups
          by Eric Ries

“translating your startup vision into a
  successful business as quickly and
        efficiently as possible”


 http://startuplessonslearned.com
Business-Technical


Technical process that touches business use

Used to guide product direction

Best when business unknowns exist
Where do you use this?

  New projects where you are unsure of your
  market

  Haven’t figured out a pricing strategy

  Don’t know what users really need

  Don’t want to write software no one uses
What is progress?


Completing stories?

Hitting project milestones?

Making profit?
None of the above?


Going in the wrong direction

More about traction over profits

Finding product-market fit
Validated learning


“Using collected data to prove key
risk factors are being addressed by
   your product's development”
Focus on adoption

Customer signups

User engagement

Long-term retention

Viral coefficient
What does FbDD look
       like?

 Don’t spec out entire project

 Keep long-term strategy in mind

 Tight iterative loops once you collect feedback
Minimum Viable Product

 “A version of a new product which
allows a team to collect the maximum
 amount of validated learning about
   customers with the least effort.”
Minimum Viable Product

 “A version of a new product which
allows a team to collect the maximum
 amount of validated learning about
   customers with the least effort.”
A starting place

Simplest implementation of your core idea

Just build enough for validated learning

Early adopters will see the potential

Not finished, should be embarrassing
Some examples



Landing page, signup page and AdWords

Rails Rumble or Startup Weekend
MVP Interaction



Questions, comments?
2. Role in development
Life of a story
An idea is born
Start with usual client-developer conversation

Ask three additional questions:

1. How do we measure progress?

2. What’s the expectation on how this story
   will impact our metrics?

3. How long do we need to measure results?
Grows up

Implement with simplest version for learning

Strive for least effort to get feedback soonest

Push off unneeded decisions, commitments

Ensure we have a way for measuring in place
Leaves home


Deploy code to production

Measure results for agreed upon duration

Though deployed, the story is not done
Makes its mark

Analyze results, good, bad or indifferent

Is it a keeper?

Possibly remove it?

Need to tweak the approach some?
The pivot

Shifting your product direction based on
validated learning

Used when progress is no longer being made

Leverage what you’ve built
Types of pivots

Segment - going after different customers

Problem - solve a different problem for existing
customers

Feature - selecting specific feature and
centering company around that
Dev loop interaction



Questions, comments?
3. Techniques

Starting techniques

 1. Tracking usage

 2. A/B testing

 3. Net promoter score

 4. Direct feedback
Tracking usage




“Know when you’re making progress”
Pirate metrics
Not vanity metrics
Actionable is key


Should help you make decisions

Gives insight into your problem domain
AARRR!!
                      by Dave McClure

          acquisition

          activation

          retention

          referral

          revenue

http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
flingr’s metrics

acquisition - site visits

activation - signups

retention - repeat use

referral - fling backs

revenue - upgrade to pro account
Drop-in service

Google Analytics (good for content/pages)

MixPanel (easy integration)

KISSMetrics (strong on funnel analysis)

StatsMix (flexible, combines many sources)
Roll your own


1. Your own event-driven metrics table

2. Use existing application data
Existing data
DIY dashboard
why DIY?

Very simple to get started

You already have this data

Easily add or change what you track

As you own the data, can analyze differently

Good enough
A/B testing

“Presenting two or more variations
 and measuring user behavior to
         determine value”
An improvement?
Anatomy of a split


           A/Bingo
        Patrick MacKenzie

http://www.bingocardcreator.com/abingo
Identify the user
Invoke a split test
Record conversion
Tracking results
Benefits


Best mechanism to measure change

Can’t be affected by outside influences

Can answer internal debates
Pitfalls


Don’t a/b test everything

Can lead to a mess if not guided by your vision

May not get conclusive results
Net Promoter Score




“How much do your users like your
          product?”
How likely are you to
  recommend ...?
Simple follow-up
What’s the point?

Lightweight way to track customer perceptions

You want to see your score improve over time

Easy way to get additional feedback

Can do small sample sizes at regular intervals
How it works

Tracks 3 groups:

 1. Promoters (9-10)

 2. Passives (7-8)

 3. Detractors (0-6)
Formula

   NPS = P% - D%
10 responses
5 promoters
3 passives
2 detractors

30 = 5/10 - 2/10
Triggering

Offer questionnaire every other week/month
or manually trigger

Only need a small sample size, such as 50-100
per time

Only present to each user once per 3-6 months
My preference

Perform small sample each week

Ask users after 3 weeks of signing up

Only ask every 6 months

Prefer in site lightbox modal to minimize
interruption
Database
Base model
Following results
Benefits


Very easy to use and implement

Not too annoying for users when done right

Allows you to track customer perception
Direct user feedback
Guidelines

Make it easy to tell you want they think

Ever present feedback button on all pages

Shouldn’t interrupt user flow

Less clicks is better
Simple version


Lightbox modal with text box and email

Start with sending results in emails

Graduate to a DB table for managing
Full featured


Get Satisfaction or User Voice

Better at collecting and managing feedback

Downside is more clicks and extra registration
Live interaction
Snap-a-bug: http://www.snapabug.com/

   Does live chat or offline email form

   Can record what’s going on with the user’s
   browser

   Integrates with existing CRMs, help desk
   and bug tracking solutions

   Advanced remote screen capture
Implementation
   interaction



Questions, comments?
4. Integration
With current process

 Plays nice with agile methodologies

 Need to consider an extra phase for learning
 once metrics are collected

 Half-implemented stories will need follow-up
 to build out
When to use
MVP: from the start

Metrics: pick 3-5 core metrics from start

A/B testing: prefer major feature rollouts or
contentious enhancements

NPS: once you get over 100 users

Direct feedback: from the start
Limitations

Challenging with low volume

Be prepared for a lack of significant results

Can’t replace vision, only give some guidance
on effectiveness
Takeaway

Feedback can be used for smarter product
development

It’s easier than you think to get started

Knowing how your customers use your
product

Can help avoid writing software no one uses
Questions? Thanks!

      Marty Haught
       @mghaught
    mghaught@gmail.com
   http://martyhaught.com

More Related Content

What's hot

What's hot (20)

Product Management
Product ManagementProduct Management
Product Management
 
Optimizely Workshop 1: Prioritize your roadmap
Optimizely Workshop 1: Prioritize your roadmapOptimizely Workshop 1: Prioritize your roadmap
Optimizely Workshop 1: Prioritize your roadmap
 
The real c in cro is communication
The real c in cro is communicationThe real c in cro is communication
The real c in cro is communication
 
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product StrategyMaking Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product Strategy
 
How to be a Good Machine Learning PM by Google Product Manager
How to be a Good Machine Learning PM by Google Product ManagerHow to be a Good Machine Learning PM by Google Product Manager
How to be a Good Machine Learning PM by Google Product Manager
 
Optimizely Workshop: Mobile Walkthrough
Optimizely Workshop: Mobile Walkthrough Optimizely Workshop: Mobile Walkthrough
Optimizely Workshop: Mobile Walkthrough
 
Lean User Testing Intro
Lean User Testing IntroLean User Testing Intro
Lean User Testing Intro
 
A/B Testing Best Practices - Do's and Don'ts
A/B Testing Best Practices - Do's and Don'tsA/B Testing Best Practices - Do's and Don'ts
A/B Testing Best Practices - Do's and Don'ts
 
Lean Metrics Riga
Lean Metrics RigaLean Metrics Riga
Lean Metrics Riga
 
Introducing: The Lab Notebook - A tool for managing a LEAN UX process
Introducing: The Lab Notebook - A tool for managing a LEAN UX process Introducing: The Lab Notebook - A tool for managing a LEAN UX process
Introducing: The Lab Notebook - A tool for managing a LEAN UX process
 
[Webinar] Innovate Faster by Adopting The Modern Growth Stack
[Webinar] Innovate Faster by Adopting The Modern Growth Stack[Webinar] Innovate Faster by Adopting The Modern Growth Stack
[Webinar] Innovate Faster by Adopting The Modern Growth Stack
 
Change is Good
Change is GoodChange is Good
Change is Good
 
Integrating the Voice of the Customer into Your Product's Development
Integrating the Voice of the Customer into Your Product's DevelopmentIntegrating the Voice of the Customer into Your Product's Development
Integrating the Voice of the Customer into Your Product's Development
 
Little Known Features of Qualtrics Research Suite That Will Make Your Life Ea...
Little Known Features of Qualtrics Research Suite That Will Make Your Life Ea...Little Known Features of Qualtrics Research Suite That Will Make Your Life Ea...
Little Known Features of Qualtrics Research Suite That Will Make Your Life Ea...
 
Testing Across the Enterprise: How Cox Automotive Scales Experimentation to M...
Testing Across the Enterprise: How Cox Automotive Scales Experimentation to M...Testing Across the Enterprise: How Cox Automotive Scales Experimentation to M...
Testing Across the Enterprise: How Cox Automotive Scales Experimentation to M...
 
Google HEART UX metrics framework
Google HEART UX metrics framework Google HEART UX metrics framework
Google HEART UX metrics framework
 
Lean UX
Lean UXLean UX
Lean UX
 
Driving Product Adoption with Insight-driven Playbooks
Driving Product Adoption with Insight-driven PlaybooksDriving Product Adoption with Insight-driven Playbooks
Driving Product Adoption with Insight-driven Playbooks
 
The Wall Street Journal - Optimizing Membership
The Wall Street Journal - Optimizing MembershipThe Wall Street Journal - Optimizing Membership
The Wall Street Journal - Optimizing Membership
 
Building High-Growth Products with Jobs-to-be-Done
Building High-Growth Products with Jobs-to-be-DoneBuilding High-Growth Products with Jobs-to-be-Done
Building High-Growth Products with Jobs-to-be-Done
 

Similar to Dev's Guide to Feedback Driven Development

How to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your BusinessHow to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your Business
Kissmetrics on SlideShare
 

Similar to Dev's Guide to Feedback Driven Development (20)

Feature Prioritization Techniques for an Agile PMs by Microsoft PM
Feature Prioritization Techniques for an Agile PMs by Microsoft PMFeature Prioritization Techniques for an Agile PMs by Microsoft PM
Feature Prioritization Techniques for an Agile PMs by Microsoft PM
 
How to Present Test Results to Inspire Action
How to Present Test Results to Inspire ActionHow to Present Test Results to Inspire Action
How to Present Test Results to Inspire Action
 
Proven Strategies for increasing Adoption and Engagement
Proven Strategies for increasing Adoption and EngagementProven Strategies for increasing Adoption and Engagement
Proven Strategies for increasing Adoption and Engagement
 
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
 
Product Management
Product ManagementProduct Management
Product Management
 
Bringing User-Centered Design Practices into Agile Development Projects
Bringing User-CenteredDesign Practices intoAgile Development ProjectsBringing User-CenteredDesign Practices intoAgile Development Projects
Bringing User-Centered Design Practices into Agile Development Projects
 
How to Use Data to Drive Product Decisions by PayPal PM
How to Use Data to Drive Product Decisions by PayPal PMHow to Use Data to Drive Product Decisions by PayPal PM
How to Use Data to Drive Product Decisions by PayPal PM
 
How to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your BusinessHow to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your Business
 
UX STRAT Europe 2021 Workshop: Jules Skopp, Expedia
UX STRAT Europe 2021 Workshop: Jules Skopp, ExpediaUX STRAT Europe 2021 Workshop: Jules Skopp, Expedia
UX STRAT Europe 2021 Workshop: Jules Skopp, Expedia
 
Building a Product Strategy by Microsoft Sr PM
Building a Product Strategy by Microsoft Sr PMBuilding a Product Strategy by Microsoft Sr PM
Building a Product Strategy by Microsoft Sr PM
 
How to Run a Data Driven Product Dev Organization by Skedulo CPM
How to Run a Data Driven Product Dev Organization by Skedulo CPMHow to Run a Data Driven Product Dev Organization by Skedulo CPM
How to Run a Data Driven Product Dev Organization by Skedulo CPM
 
0 to 1- Taking a Concept & Launching an MVP by Wish Product Leader
0 to 1- Taking a Concept & Launching an MVP by Wish Product Leader 0 to 1- Taking a Concept & Launching an MVP by Wish Product Leader
0 to 1- Taking a Concept & Launching an MVP by Wish Product Leader
 
Building a Product Strategy by Microsoft Sr PM
Building a Product Strategy by Microsoft Sr PMBuilding a Product Strategy by Microsoft Sr PM
Building a Product Strategy by Microsoft Sr PM
 
Measuring What Matters by Google PM
Measuring What Matters by Google PMMeasuring What Matters by Google PM
Measuring What Matters by Google PM
 
How to Identify Relevant Product KPIs by Roomgo Head of Product
How to Identify Relevant Product KPIs by Roomgo Head of ProductHow to Identify Relevant Product KPIs by Roomgo Head of Product
How to Identify Relevant Product KPIs by Roomgo Head of Product
 
Conversion Optimization Framework to Build Sustainable and Repeat Growth
Conversion Optimization Framework to Build Sustainable and Repeat GrowthConversion Optimization Framework to Build Sustainable and Repeat Growth
Conversion Optimization Framework to Build Sustainable and Repeat Growth
 
User-Driven Development: The Art of Creating a Software Product - Velvetech
User-Driven Development: The Art of Creating a Software Product - VelvetechUser-Driven Development: The Art of Creating a Software Product - Velvetech
User-Driven Development: The Art of Creating a Software Product - Velvetech
 
Moving from Collaboration Pilot to Successful Implementation
Moving from Collaboration Pilot to Successful ImplementationMoving from Collaboration Pilot to Successful Implementation
Moving from Collaboration Pilot to Successful Implementation
 
A/B Testing for New Product Launches by Booking.com Sr PM
A/B Testing for New Product Launches by Booking.com Sr PMA/B Testing for New Product Launches by Booking.com Sr PM
A/B Testing for New Product Launches by Booking.com Sr PM
 
Clover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationClover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive Experimentation
 

Recently uploaded

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Recently uploaded (20)

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 

Dev's Guide to Feedback Driven Development

Editor's Notes