This presentation is an Introduction to the importance of Data Analytics in Product Management. During this talk Etugo Nwokah, former Chief Product Officer for WellMatch, covered how to define Data Analytics why it should be a first class citizen in any software organization
4. Data analytics should be a first-class citizen
Importance of Data Analytics
Data analytics team should
be a key stakeholder
Everyone should ‘own’ the data
10. My first exposure to Data Analytics
for Product Management
2004 2017
11. Data analytics refers to qualitative and
quantitative techniques and processes
used to enhance productivity and
business gain.
12. Data Analytics is not:
Big Data
Artificial Intelligence (“AI”)
or Machine Learning
Data Science
13. Data Science is a concept that unifies
statistics, data analysis and their related
methods in order to understand and
analyze phenomena with data.
It employs techniques and theories drawn
from many fields within the broad areas of
mathematics, statistics, information
science, and computer science.
14. Big Data is defined as high-volume, high-
velocity and high-variety information
assets that demand innovative forms of
information processing for enhanced
insight and decision making.
1012
15. Data Science has several sub-domains
MachineLearning
Visualization
DataMining
ClusterAnalysis
Classification
16. Research and extract information from
various sources to describe the main features
of a data set in support of business operations
and decision making.
Data Analyst/Journalist
Data Scientist
Cleanse data, apply various techniques to
large data sets and create visualizations to
understand to answer more abstract business
questions, provide more predictive insights
and anticipatory behaviors.
23. Liquidity
Breaking down
silos of data
Structural - applications are
optimized for their main function,
not to encourage data sharing
Political - Knowledge is power
Growth - Company evolution
results in different
approaches to data
Vendor - Incentivized to store
data exclusively
29. Organizational Structure - Example
+
Dir of
Product
CPO
Sr Director of
UI/UX
Dir of
Analytics
CEO
Leadership
Team
30. Ensure someone is representing data
on your agile/scrum team
PMPM
UI/U
X
UI/U
X
EngEng
DataData QAQA
Build metrics into user stories to measure success
Build tags into features
Identify R&D opportunties
34. What is this the
problem?
How big is the
problem?
How do we know if
we’ve solved it?Julie Zhou, VP Product Design
Approach to building new
features
Example use of analytics
35. //
Referral
Acquisition
Activation
Retention
# of app downloads
#of registrations
% of users registered
# of physician reviews
# of searches performed
app reviews
Net Promoter Score
Referral Emails
# of unique return users per day
# of sign-ins per month per cohort
% of conversion to desired behavior
Example
Metrics
The funnel is a critically important framework
37. How we used it: Across web and apps, understand
effectiveness of customer campaigns and cursory user
patterns.
Description: Powerful analytics tool describes what users are
doing and how they got to your site/app, but not why they are
behaving a certain way.
//
Acquisition
38. How we used it: Determined primary user focus on homepage
Description: Heat map, scroll maps and click tracking to
understand user focus and what draws their attention
//
Acquisition
39. How we used it: Across web and apps developed deeper
understanding of how users engaging per page and through the
conversion funnel
Description: Behavioral analytics and engagement tool
//
Activation
Retentio
n
40. How we used it: Built a set of custom metrics in our admin tool that
looked similar to the kiss metrics UI interface but more dynamic
Description: set of APIs that allow you to stream, compute,
and visualize events
//
Activation
Retentio
n
41. //
Activation
Retentio
n
Acquisition
How we used it: Funnel analysis, cohort analysis, click behavior
Description: A click-stream technology that auto-tags all of the
activity on your product to enable analytics and insights
42. How we used it: To produce ROI reports and custom reports for our
large employer customers
Description: Data visualization tool focused on business
analytics
44. Help your organization demonstrate the
5 data-driven characteristics
Learn the latest data analytics tools and
methods in the industry
Find the data analytics team
in your organization and make friends
If you don’t have a data team,
be the Lorax for the data
Etugo’s tips:
I am made up of a patchwork of different experiences:
Product Executive with a background in Industrial Engineering, Consultant and also Painter/Artist
•In 1999, a young company based in Los Gatos, California with less than 20 employees and on the edge of going bust. They had a couple experienced co-founders but the problem was that they were stuck at about 300,000 customers. They were essentially providing the same general pay-per-rental movie experience as other big box movie rental stores.There were, as always, some early adopters but generally people didn’t seem very willing to change. The team knew that the service wasn’t better enough to get people to change. Even worse, DVD sales were starting to lag, and a Hollywood backlash further muddied the situation. Then there were challenges with fulfillment logistics, difficulty maintaining DVD quality, and trying to figure out how to do all this in a way that covered costs and generated some cash. There was a product manager on the team knew they needed to do something different.
One of many tests they tried was to move to a subscription service. Get people to sign up for a month and offer them unlimited movies. Would that be perceived as “better enough” to get them to change their media consumption behavior? The good news was that yes, actually, this really did appeal to people. A flat monthly fee and all the videos they could consume sounded pretty awesome. The bad news is that the team created some real problems for themselves. No surprise that Netflix customers wanted to rent mostly newly released feature films, yet these were much more expensive for Netflix to stock, and they would need to stock so many copies of these, that they’d very likely run out of money fast. So the product challenge became how were they going to make sure Netflix customers could watch a set of movies they would love, yet wouldn’t bankrupt the company?
They knew they needed to somehow get customers to want a blend of expensive and less expensive titles. Necessity being the mother of invention, this is where the queue, the ratings system, and the recommendation engine all came from. Those were the technology-powered innovations that enabled the new, much more desirable business model.
Between working with the co-founders on the strategy, validating concepts with the users, assessing the analytics, driving features and functionality with the team, and working with finance on the new business model, marketing on acquisition, and the warehouse on fulfillment, you can imagine the workload Kate faced on a daily basis.
Every time you go shopping, you share intimate details about your consumption patterns with retailers. A columnist for the New York Times wrote about this by taking to a Target statistician Andrew Pole -- before Target freaked out and cut off all communications. about the clues to a customer's impending bundle of joy. Target assigns every customer a Guest ID number, tied to their credit card, name, or email address that becomes a bucket that stores a history of everything they've bought and any demographic information Target has collected from them or bought from other sources.
[Pole] ran test after test, analyzing the data, and before long some useful patterns emerged. Lotions, for example. Lots of people buy lotion, but one of Pole’s colleagues noticed that women on the baby registry were buying larger quantities of unscented lotion around the beginning of their second trimester. Another analyst noted that sometime in the first 20 weeks, pregnant women loaded up on supplements like calcium, magnesium and zinc. Many shoppers purchase soap and cotton balls, but when someone suddenly starts buying lots of scent-free soap and extra-big bags of cotton balls, in addition to hand sanitizers and washcloths, it signals they could be getting close to their delivery date.
My daughter got this in the mail!” he said. “She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?” The manager didn’t have any idea what the man was talking about. He looked at the mailer. Sure enough, it was addressed to the man’s daughter and contained advertisements for maternity clothing, nursery furniture and pictures of smiling infants. The manager apologized and then called a few days later to apologize again. On the phone, though, the father was somewhat abashed. “I had a talk with my daughter,” he said. “It turns out there’s been some activities in my house I haven’t been completely aware of. She’s due in August. I owe you an apology.”
https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/2/#65e63b047e85
Among the bad publicity Uber has got lately from sexual harrassment in the workplace to stealing IP, the one that caught my eye was related to how they used data analytics.
Now it's a story in the New York Times about how Uber dodged authorities worldwide where law enforcement and other officials were running sting operations when Uber wasn't authorized to operate. Using data from the app and other unnamed sources, the company would identify who was working for the government, cancel any rides they ordered, and put up a faked screen of where vehicles were at the time.
Source: https://903ink.onlinebusiness.american.edu/blog/comparing-analytics-data-science/
Predictive, Prescriptive
Interconnected but different disciplines. The lines between analytics and data science can definitely be very blurry.
https://en.wikipedia.org/wiki/Data_science
Source: http://www.sciencedirect.com/science/article/pii/S0268401214001066
One terabyte is considered to be big data. One terabyte stores as much data as would fit on 1500 CDs or 220 DVDs, enough to store around 16 million Facebook photographs. Beaver, Kumar, Li, Sobel, and Vajgel (2010) report that Facebook processes up to one million photographs per second. One petabyte equals 1024 terabytes. Earlier estimates suggest that Facebook stored 260 billion photos using storage space of over 20 petabytes.
Variety refers to the structural heterogeneity in a dataset. Technological advances allow firms to use various types of structured, semi-structured, and unstructured data.
Velocity refers to the rate at which data are generated and the speed at which it should be analyzed and acted upon.
Statistics and modeling; training your model to recognize and anticipate patterns in the data.
Real-time vs non-real time; R&D vs Operational
McKinsey Global Institute found that data-driven organizations are 23 times more likely to acquire customers, six times as likely to retain those customers, and 19 times as likely to be profitable as a result. http://www.ey.com/Publication/vwLUAssets/EY-global-becoming-an-analytics-driven-organization/%24FILE/ey-global-becoming-an-analytics-driven-organization.pdf - E&Y found that surveying thousands of
https://www.entrepreneur.com/article/237326
Metrics define your corporate culture. More than anyone I’ve ever met, Bezos knew that things don’t improve unless they’re measured.
A data-driven culture is meaningless without the support of the CEO and executive team -- and their own willingness to challenge assumptions that they hold dear.
While I was at Amazon, my team proved (65 pages of proof, in fact!) the impact of website load times on sales, and identified the metrics that mattered. Without hesitation, Bezos and Amazon reorganized around a new, even more aggressive method of measuring website performance, changing hundreds of jobs to obsess about these very metrics.
https://hbr.org/2016/12/breaking-down-data-silos
Since the popular emergence of data science as a field, its practitioners have asserted that 80% of the work involved is acquiring and preparing data.
Structural. Software applications are written at one point in time, for a particular group in the company. In a world of limited resources, applications are optimized for their main function. The incentives of individual teams are unlikely to encourage data sharing as a primary requirement. This focus on function, for instance, may result in recent sales being stored in different systems from historical sales, thus presenting an immediate barrier to boosting sales through personal product recommendation.
Political. Knowledge is power, and groups within an organization become suspicious of others wanting to use their data. And often with some justification, as the scope for misuse, even accidental, is broad. Data isn’t a neutral entity — you must interpret it with knowledge of its history and context. This sense of proprietorship can act against the interests of the organization as a whole.
Growth. Any long-lived company has grown through multiple generations of leaders, philosophies, and acquisitions, resulting in multiple incompatible systems. Even if there are no political issues in integrating data, it is costly to reconcile and integrate sets of data that embody different approaches to important business concepts.
Vendor lock-in. Software vendors are among the first to know that access to data is power, and their strategies can frustrate the desire of users to export the data contain in applications. This is particularly dangerous with software-as-a-service applications, where the vendor wants to keep you within their cloud platform. Vendors have also worked hard to create entire job functions and career paths centered around their software. Any hint of move from that world could threaten the livelihood of a trained and certified software professional.
Folks like Larry Page, Elizabeth Warren
Apple typically does development
Avoid the tension with data science/engineering. Ideally this should be the same team.
The product manager is like the Lorax, speaks for the trees. The PM speaks for the customers, the users, others in the organization and the most importantly with data!