O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

How to Use Data to Build Products by Tradesy Product Advisor

Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Carregando em…3
×

Confira estes a seguir

1 de 42 Anúncio

How to Use Data to Build Products by Tradesy Product Advisor

Baixar para ler offline

In this presentation:
-Product Management is probably the most exciting function in technology organizations - it's an art and science that's well-suited for certain personalities
-The goal of a good Product Manager is NOT to launch a product - rather, it's to move a planned metric in the right direction by the right amount
-A good Product Manager can answer the question, "How did your product do yesterday?" We can't answer that without a well-defined analytics strategy and data requirements built into our products

In this presentation:
-Product Management is probably the most exciting function in technology organizations - it's an art and science that's well-suited for certain personalities
-The goal of a good Product Manager is NOT to launch a product - rather, it's to move a planned metric in the right direction by the right amount
-A good Product Manager can answer the question, "How did your product do yesterday?" We can't answer that without a well-defined analytics strategy and data requirements built into our products

Anúncio
Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a How to Use Data to Build Products by Tradesy Product Advisor (20)

Anúncio

Mais de Product School (20)

Mais recentes (20)

Anúncio

How to Use Data to Build Products by Tradesy Product Advisor

  1. 1. www.productschool.com How to Use Data to Build Products by Tradesy Product Advisor
  2. 2. FREE INVITE Join 30,000+ Product Managers on
  3. 3. COURSES Product Management Learn the skills you need to land a product manager job
  4. 4. COURSES Coding for Managers Build a website and gain the technical knowledge to lead software engineers
  5. 5. COURSES Data Analytics for Managers Learn the skills to understand web analytics, SQL and machine learning concepts
  6. 6. COURSES Digital Marketing for Managers Learn how to acquire more users and convert them into clients
  7. 7. COURSES Blockchain for Managers Learn how to trade cryptocurrencies and build products using the blockchain
  8. 8. David B. Miller TONIGHT’S SPEAKER
  9. 9. Product Management, Data Analytics/Business Intelligence Frameworks David B. Miller
  10. 10. After Today, You’ll Have Learned More About... 1. My product management framework that describes the function of product management and outlines the role/responsibilities from concept through launch to business success. 2. Deeper dive into data analytics/business intelligence: Best practices for making informed decisions and effecting positive business impact.
  11. 11. A company needs “product vision,” but doesn’t “need” a product manager ● Goal of any company is to grow revenue profitably ● Companies need to: ○ Build ○ Sell ● A company can consist of a single “heavy lifter” who has the product vision and can both build and sell How do you Build well? How do you Sell well? ● Decide what to build (i.e., plan) well ● Execute well ● Market well (all four of those P’s) $ $$ (+) Return on Investment (ROI) is when the output is larger than the input
  12. 12. As complexity increases, Product Managers reduce risk and increase the probability of success ● A product manager manages the… ○ Planning ○ Execution/Production ○ Business success ....of the product ● Mini-CEO of the product ● E2E accountability for product success - from product vision through to meeting post-launch business goals Good product managers:1) Bring clarity to confusion; 2) Exercise good business judgment; 3) Make problems “go away.”
  13. 13. Product Management Framework - PM as Accountable Owner Business Goals Product's true north in ever- changing world. Reason for being. Prove product will appeal to a large enough audience Define product success (Launching the product is NOT product success) Prioritize product planning based on business goals and level of effort Define must- have and nice- to-have features for any given product version The heavy lifting - getting the product launched: With the right features; On time; On budget Ensure the product is meeting the agreed upon business goals/ KPIs Wash, rinse, repeat Kaizen: Continuous Improvement Product Vision Product & Market Strategy Business Model & Business Goals Product Roadmap Product Req’ts Execution Product Success Product Development Process
  14. 14. Product Vision ● The product’s “reason for being” - it’s why everyone’s together working to get it launched ● It MUST solve a user/market need for a certain “customer” ● The product REALLY SHOULD solve it in a way that’s better than the competition. Strive for product excellence. Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process Emotion Unexpected Core Credible Do users react with visible positive emotions? Do they smile? Will the product be one of many or stand out? A user should be amazed/ excited/ jump out of their skin. What is the one thing this product will do really well? All features that support this are essential - everything else is nice to have. Will users believe you’ll have the ability to solve their problem? Snapchat iPod’s clickwheel Tesla Amazon Web Services (AWS) Contribution: Made to Stick: Why Some Ideas Survive and Others Die. Book by Chip Heath and Dan Heath
  15. 15. Product Vision ● Needs to cover the three ingredients below: Target Customer Needs Addressed Vis-à-vis Competition ● Who are they? ● Be specific ● Is the market large enough to be compelling? ● Focus on the most critical ● Better to do one thing well than many things mediocre ● What makes it unique? ● Is there sustainable competitive advantage? Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process
  16. 16. Product Vision ● The vision REALLY SHOULD be bold/exciting/motivating/inspiring/ambitious. Boldness attracts both customers and talent. Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process Bad Good Build kick-ass products SpaceX: Actively develop the technologies to make space exploration possible, with the ultimate goal of enabling human life on Mars. Amazon: Launch products that make us the earth's most customer centric company; to build a place where people can come to find and discover anything they might want to buy online.
  17. 17. Good Product Vision Requires Strategic Fit vis-à-vis the Competition ● Companies can compete on: ○ Price ○ Quality (how well you meet user needs) ● Where do you fit? ● How crowded? ● Does it make sense? Is it credible? Attribute 1 Low High Attribute 2 Low High Them Them Them Us Illustrative Example Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process
  18. 18. Product and Market Strategy ● Product/Market Fit: Degree to which your product is satisfying market/user demand ● Choosing the right market is often more important than having the best product ● Solid Go-to-Market (GTM) strategy is crucial for making the product <> market connection Good Product Bad Product Big Market Small Market Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process Sweet Spot Short-term success if unique, but need to improve product vs. competition to be sustainable Not worth it unless investment is really, really low Need to adjust/pivot, else please just quit and go home
  19. 19. Business Model & Business Goals ● Companies are not schools or charities - goal of any product is to get a return on investment for investors/shareholders ● Revenue is a lagging indicator of success, so not a great success metric Possible Leading Indicator of Success Why Is This Important? How Do You Measure This? Total downloads Assesses marketing awareness From various app stores and/or your hosting servers Total number of users Assesses usage From registrations or your definition of a “user/device” Daily / Monthly active users (DAUs / MAUs) Assesses how important your product is to users # of users who “engaged” in a day/month. # of sessions per DAU Assesses “social-ness” / how often users are returning to your product in a day [(# daily sessions) / DAUs] Time spent per session Assess engagement of your users [(sum of session time) / (# daily sessions)] Stickiness Assesses the activity / engagement of your users [ DAUs / MAUs ] XX (i.e., 30) day retention Assesses how important your product is to users [(# active users in “cohort” now) / (# active users in “cohort” before)] Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process eCommerce: Conversion rate, Average order value, Repeat customer rate, CLTV, Funnel abandonment
  20. 20. Business Model & Business Goals ● Choosing the right leading indicators of success - your Key Performance Indicators (KPIs) - is art & science ● If your KPIs are growing and healthy, then revenue will follow ● Need to ensure that the product is “instrumented” in a way such that these KPIs can indeed be measured ● Success is not simply launching the product - Need to plan and agree upon the success metrics of the launch ahead of time: ○ What KPIs do we intend to move? ○ By how much? ○ By when? Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process
  21. 21. Product Roadmap ● What products/features/versions/releases you’re going to launch when ● Document that lives in a certain place and represents the single source of truth (SOT) ● All roadmapped products MUST align with the Product Strategy ● $#!+ happens, things change - so roadmaps are updateable ● Tradeoffs will need to happen based On (changing) business priorities Q1 Q2 QN v1 ● Feature 1 ● Feature 2 ● Feature 3 v1.1 ● Feature 1..N v2 ● Feature 1..N Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process
  22. 22. Product Requirements ● For any given product launch/version/release/etc., a document - often referred to as a Product Requirements Doc (PRD) - that specifies for the engineering team EXACTLY “what” the product shall do ● Product Management specifies the “what,” and engineering - via their engineering design documentation - specifies the “how” ● Product requirements are documented in a single location [choose_your_system], and they do NOT live in email, text, hallway conversations, etc. ● When the engineering team has any questions about the PRD, it means the PRD was unclear. Clarifications are NOT answered via email or phone, but rather via a revised version of the PRD. Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process
  23. 23. Product Requirements Shall Be 100% Unambiguous Poorly Specified Better When the current time reaches the alarm time, the alarm goes off ● What does “off” mean? When the user hits the snooze button, the alarm should pause for 5 minutes. ● What does “hit” mean? ● What does “should” mean? ● What about the alarm “pauses?” Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process When the current time reaches pre- set alarm time, the alarm clock begins to make a noise that’s loud enough to awaken the average sleeper When the user depresses the snooze button, the alarm noise stops for 5 minutes and then resumes
  24. 24. Product Requirements Shall Be Prioritized ● No company has infinite resources and there are only 24 hours in a day ● $#!+ happens, things change, etc. So not everything will “fit” into the release. Between Quality / Time / Cost, you can usually only fit two. ● As much as possible, quantify upside through testing and/or market research ● Align features with both product vision and business KPI goals ● Prioritize by “must haves” and “nice to haves,” and “must haves” are the launch blockers Low Level of Effort (LOE) High LOE Low Upside High Upside ROI ≈ [Upside / LOE] Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process No brainer / low hanging fruit Likely to be deprioritized Depends Depends. Sometimes low hanging fruit.
  25. 25. Execution (Git ‘Er Done) ● Sometimes PM is assisted by project/program management (PjM/PgM) ● Often involves daily “stand-ups” ○ Green = No blockers. Great, stop talking, move onto the next item. ○ Yellow = At risk. What help is needed to make you green? ○ Red = Help needed. Take offline and management to unblock / remove obstacle / provide the necessary help. ● Consider “pilots” or experiments as a way to get through Fear/ Uncertainty/ Doubt (FUD) ● Run to trouble. Get in front of something and control it before it controls you. Letting Things Happen Making Things Happen I sent an email, and I haven’t heard back... I went to the person directly and... I told Jill and Jack that they should talk... I scheduled a meeting with Jill and Jack to discuss... Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process
  26. 26. Product Management Is the Accountable Party ● The accountable party can sub-contract to the responsible party. (e.g., PM needs Legal to conduct an Intellectual Property (IP) review.) ● The accountable party can also be the responsible party. (e.g., PM documents product requirements.) Accountability Responsibility ● On the hook to ensure something happens ● Single throat to choke ● On the hook to physically execute a task or action item ● Heavy lifter Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process
  27. 27. Product Success ● Congrats, you’ve launched the release, which is only step 1 ● Are you hitting your KPIs? ○ Yes? Great, double-down and do more. ○ No? Why not? What’s your hypothesis and what test(s) can you perform to prove it and remediate? Product Vision Product & Market Strateg y Bus. Model & Goals Road map Req’ts Exec. Succ -ess Product Development Process Product Vision Product & Market Strategy Business Model & Business Goals Product Roadmap Product Req’ts Execution Product Success Product Development Process Wash, rinse, repeat. Kaizen: Continuous Improvement.
  28. 28. Data Is the New Oil ● Information is not free ○ Considerable cost of building data pipelines ○ Non-zero cost of pulling data and analyzing it ○ Opportunity cost ● Information is only valuable if it: ○ Reaffirms a decision ○ Changes a decision ● Answers, not Data! ○ Data is a means to end ○ What’s the question behind the question? PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  29. 29. Not All Data/Information is Valuable Hypothetical Situation / Information Gathered Resulting Action Taken by Organization Information Was Valuable? Someone requested a report of daily revenue over the last 90 days Yes – revenues were lower than expected so the org had to take remedial measures There was concern that the org was falling short on revenue, so daily revenue over the last 90 days was compiled None – revenue looked healthy Someone was curious as to what daily revenue had been over the last 90 days None Yes Yes – The org would have taken remedial measures if the data had supported the initial concern No PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  30. 30. Be Protective of Requests for “Data” ● Data requests need to be well-founded and formulated ● The proposed analysis needs to actually be able to answer the question at hand ● Analysis/information will indeed be valuable to the organization Tenet Why Is This Important? Best Practice What’s the value of this information? Information is a product. Regardless of whether it’s ever sold externally, we must be conscious of why it’s being produced. Like any well-planned product, it needs to also have an accompanying plan. If no solid plan can be erected, then it’s possible this information isn’t actually valuable. ● What actions will we take based on the various possible outcomes of this analysis? ● Document and hold agreed- upon owners accountable PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  31. 31. Tenet Why Is This Important? Best Practice No analysis for analysis sake Reports/analyses are not free. Just because team XYZ has been running report ABC for the last two years does not mean that report has any value. Especially if it’s being shipped to its normal distribution list and nobody is either reading it or taking any action. ● Audit what reports are or aren’t being used. Consider assessing the value of the information per above. ● If in doubt, consider terminating a report and seeing if anyone “jumps up and down.” Curiosity is not a business priority This happens frequently in meetings: “It would be interesting to take a look at XYZ.” Or, “I’m curious to see what ABC looks like.” Nonetheless, as mentioned above, compiling information with no clear plan and/or no intended actions is a waste of resources. “Let’s be really clear about whether we want to run that analysis or whether we should do something of higher priority. If we’re not resourced or planning to take action on that, then it’s possible it wouldn’t be a high priority.” Be Protective of Requests for “Data” PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  32. 32. Tenet Why Is This Important? Best Practice Business questions and business answers, not data There is always a “question behind the question.” For example, “Give me daily orders for the last 90 days” is NOT the ultimate business question – it’s simply a data request. Oftentimes the data that’s being requested is actually not the best way to answer the true underlying question. Ultimately, the deliverable is NOT the actual data or analysis. Rather, it’s the answer to the original business question. The underlying analysis should accompany/support the answer to the business question with an explanation of why the data proves/disproves the hypothesis. Take a consultative approach similar to next slide Be Protective of Requests for “Data” PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  33. 33. Be Protective of Requests for “Data” “Hey, give me daily orders for the last 90 days.” “What question are you ultimately trying to answer?” “I’d like to know whether there’s a problem with our order system.” “Oh, we have order error logs running over here. They’re extremely accurate. That’s a better way to answer your question.” “OK I ran a report from the order system error logs. The quick answer to your question is “No, there is not a system problem.” I created a slide that reiterates my assessment and shows the data that led me to that conclusion.” PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  34. 34. Tenet Why Is This Important? Best Practice Averages can lie Don’t be fooled by taking the average of two sets, comparing them, and making an assessment that one set is higher than the other. Statistically speaking, they could be the same. Run a statistical analysis to understand whether the differences in the averages is statistically significant Precise but not accurate Don’t include unnecessary decimal places in any analysis, as it can falsely imply exactitude 99.9% of the time one decimal place is enough :-) Be Protective of Requests for “Data” PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  35. 35. Tenet Why Is This Important? Best Practice If you don’t document it, it didn’t happen As mentioned, producing analysis is the same as producing any product. We don’t throw products away, and same goes for analyses. All too frequently analyses are pasted into email, sent, and then lost in the ether. Meanwhile, if someone asks the same question later, the same/similar analysis is often redone. Treat your analyses like first class products and archive them in an accessible repository. Teach customers to search through the repository first before making a new request. Be Protective of Requests for “Data” PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  36. 36. You Shall Have a Single Source of Truth (SOT) For Your Data ● “Single” ○ Information / data / documents shall only live in one place ○ Keeping the same information in multiple locations forces org to keep multiple sources updated and in synch, which creates a burden/tax ○ Inevitably, the information will be out of synch, calling into question the accuracy of all sources. ● “Truth” ○ The data shall be accurate, else it’s garbage in, garbage out. ○ Data governance: Ensure different teams are defining data / fields / etc. in the same way. Document the lexicon / data dictionary and make it accessible. ○ Audit the quality of your data by running tests with known outcomes PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat.
  37. 37. Did Your Product Have a Good Day Yesterday? PV P&MS BM& BG PR PR Ex PS Product Development Process Wash, rinse, repeat. ● If you can’t answer this, it means you’re flying blind ● What are your leading indicators of success? (Hint: It’s not revenue) Gardener Product Manager 1. Wake up 2. Tend to the plant 3. Assess and mend as-needed 4. Repeat 1. Wake up 2. Tend to the product 3. Assess and mend as-needed 4. Repeat
  38. 38. Backup
  39. 39. Product Management Framework - PdM as Accountable Owner Business Goals Product's true north in ever-changing world. Reason for being. Prove product will appeal to a large enough audience Define product success (Launching the product is NOT product success) Prioritize product planning based on business goals and level of effort Define must-have and nice-to-have features for any given product version The heavy lifting - getting the product launched in time Ensure the product is meeting the agreed upon business goals/ KPIs PdM Deliverables Market Requirements Doc (MRD) Product Roadmap Product Requirements Doc (PRD) incl Minimal Viable Product (MVP) Execution Examples & non-PdM Deliverables Product Evangelism Stakeholder Motivation & Influence Innovation/ Ideation Product/Mkt Fit Competitive Landscape Business Case Business KPI Goals and Timing AOP Planning Make hard tradeoffs between strategy, KPIs, cost, level of effort, user experience, business risk, and timing Project Plan Sprints and standups to hit project plan Marketing Plan Monitor KPIs and modify product & planning for improvement PdM Partners/ Stakeholders Strat Dev, Marketing, Design Strat Dev, Corp Strat, Marketing, Consumer Insights Finance Everyone Marketing, Design, Mfg, Software Dev, Strat Dev, QE, Packaging, Privacy/Data Security, Finance, Legal Marketing, Design, Finance Product Vision Product & Market Strategy Business Model & Business Goals Product Roadmap Product Req’ts Execution Product Success Product Development Process
  40. 40. Competitive Advantage Market Small Large Competitive Advantage Lower Cost Focus on having “me-too” products with “sufficient” features but at a lower cost Be the lowest cost operator in the industry Better Product (i.e., Product Differentiation) Focus on niche markets where product has custom/tailored features for that niche Have the absolute best product out there, for which you can charge a premium
  41. 41. Waterfall vs. Agile/Scrum Waterfall (Traditional) Agile/Scrum Sequential and well structured Adaptable and flexible ● Plan a huge / monolithic release with known features, from concept to launch ● All requirements are known ahead of time ● Plan out the start and end dates ● Iterative bits/pieces of concept to launch ● All requirements are not known ahead of time ● Final end date is not known
  42. 42. www.productschool.com Part-time Product Management, Coding, Data, Digital Marketing and Blockchain courses in San Francisco, Silicon Valley, New York, Santa Monica, Los Angeles, Austin, Boston, Boulder, Chicago, Denver, Orange County, Seattle, Bellevue, Toronto, London and Online

Notas do Editor


  • Boldness attracts both customers and talent

  • Boldness attracts both customers and talent
  • Does it make sense? Strategic dimensions.
  • Churn rate = 1 - Retention rate
    eCommerce: Conversion rate, Average order value, Repeat customer rate, CLTV, Funnel abandonment
  • Information shall not live in more than one location
    Product management shall not out-engineer the engineers
  • Focus on the word “shall” vs. “should”
  • It helps all the gears and cogs run well. It’s a valuable asset unto itself.
  • What’s the ROI of answering this question?
  • Two key words: Single and Truth. Page views and ad requests.

×