SlideShare uma empresa Scribd logo
1 de 14
Baixar para ler offline
Making sense of analytics for
documentation pages
Kumar Dhanagopal
API The Docs Virtual 2023 - Feedback, Metrics, Analytics
TOPICS
What analytics is and isn’t
Overview of key metrics
The analytics process
Interpreting metrics
Summary
2
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
How analytics can (and can’t) help…
✓ Understand how users engage with our content
• Who and from where
• How often
• On what devices
• …
✓ Understand user behavior on our site
• How they navigate
• What they click
• Where they spend time
• …
• Analytics can’t help us learn about user satisfaction and sentiment
• What do users need?
• Did they find it?
• Were they satisfied?
• How (and how much) did they read?
• Need other sources (e.g., user ratings, comments)
• …
3
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
Data != reality (correlation doesn’t imply causation)
4
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
The key metrics
5
A B C
D E
F G H
Page Views Visits Entries Exits Bounces
A 1 1 0 0 0
B 2 2 0 0 0
C 1 1 1 0 0
D 3 2 0 0 0
E 1 1 1 1 1
F 2 1 1 0 0
G 2 2 0 1 0
H 1 1 0 1 0
Session-1
Session-2
Session-3
Note:
❖ Repeat visits and long visits are treated differently
❖ Cookie’s matter
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
The analytics process
1. Define the problem or goal
2. Get the required data
• Metrics
• Dimensions
• Period
3. Prepare the data
• Remove noise
• Fix inconsistencies
• Aggregate data
4. Analyze, explore, visualize
5. Describe, diagnose, prescribe, predict
6
"If you torture the data long enough, it will confess – to anything" – Ronald H. Coase
Minimize the garbage-in-garbage-out risk.
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
Interpreting metrics: General guidelines
• Validate with other data
• Avoid cross-product comparisons
• Look at trends, not just absolute numbers
• Remember: offline and second-hand are not tracked!
• Consider the content type, length, format, and structure
• Use analytics data as supplementary input for decision making
7
Page Views
A 15k
B 14k
Jan Feb Mar Apr May Jun
3358 2610 3025 2411 2316 1900
0 0 1583 3308 5015 5029
Page A 25k 25k
Page B 25K 25k
Page C -- 25k
Page D -- 25k
Chunking affects analytics!
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
Doc 1 Doc 2
50k views 100k
Interpreting low page views
• Popularity != page views
• Look at trends, not just absolute numbers
• Consider the “age” of the page
• Check whether the page is discoverable
• Keep in mind structural changes
• Don’t ignore zero-view pages
8
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
Interpreting bounce rate
• Possibly the most misunderstood metric
• High bounce rate is not necessarily a problem
• Text-book guidance might not apply to
documentation websites
• Session time-out = bounce
• Analyze in conjunction with other metrics, like time
spent on page
9
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
Interpreting time spent on page
• High time spent:
• Was the content useful/interesting?
• Difficult to understand?
• User left the browser open too long? ☺
• High time spent + high page views
• Too much content, or too complex?
• Opportunity to improve chunking
• High time spent + high bounce rate
• Landing pages: cause for concern
• Other pages: Opportunity to simplify content
• Low time spent + high bounce rates
• Reference pages: positive indicator?
10
Time spent is NOT calculated for exit pages!
Number
of users
Total time
spent (min)
Avg. time
spent (min)
Action after
reading…
Group A 50 500 10.0 Exit site
Group B 50 250 5.0 Another page in
the site
Total 100 750 7.5
2.5
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
Analytics “targets”
Should we have analytics KPIs?
Documentation sites != e-commerce portals
Consider data availability and decision-making culture
11
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
SUMMARY
✓ Data != reality
✓ Analysis requires non-trivial effort
✓ Metrics won’t give us all the answers
✓ Focus on users, business goals
✓ The forest AND the trees matter!
12
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
SUGGESTED READING
Web Analytics - An Hour A Day, Avinash Kaushik
Practical Text Analytics, Steven Struhl
An Introduction to Data Science, Jeffrey S. Saltz, Jeffrey Stanton
13
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
THANK YOU!
https://www.linkedin.com/in/kumardhanagopal
14
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

Mais conteúdo relacionado

Semelhante a Making sense of analytics for documentation pages

Semelhante a Making sense of analytics for documentation pages (20)

Analysing content beyond google analytics
Analysing content beyond google analyticsAnalysing content beyond google analytics
Analysing content beyond google analytics
 
Harnessing the Value of UX
Harnessing the Value of UXHarnessing the Value of UX
Harnessing the Value of UX
 
Social media data analysis
Social media data analysisSocial media data analysis
Social media data analysis
 
Dlf 2012
Dlf 2012Dlf 2012
Dlf 2012
 
Customer validation with Diary Studies
Customer validation with Diary StudiesCustomer validation with Diary Studies
Customer validation with Diary Studies
 
Using analytics in ux design my view
Using analytics in ux design   my viewUsing analytics in ux design   my view
Using analytics in ux design my view
 
Content Strategy and User Experience: Combination in Practice
Content Strategy and User Experience: Combination in PracticeContent Strategy and User Experience: Combination in Practice
Content Strategy and User Experience: Combination in Practice
 
Combining Methods: Web Analytics and User Testing
Combining Methods: Web Analytics and User TestingCombining Methods: Web Analytics and User Testing
Combining Methods: Web Analytics and User Testing
 
Master Conversion Optimization
Master Conversion OptimizationMaster Conversion Optimization
Master Conversion Optimization
 
Building Your Analytics Toolbox
Building Your Analytics ToolboxBuilding Your Analytics Toolbox
Building Your Analytics Toolbox
 
SPSCT15 - An Independent Evaluation of Third-Party SharePoint Analytics Offer...
SPSCT15 - An Independent Evaluation of Third-Party SharePoint Analytics Offer...SPSCT15 - An Independent Evaluation of Third-Party SharePoint Analytics Offer...
SPSCT15 - An Independent Evaluation of Third-Party SharePoint Analytics Offer...
 
Lean Analytics & Analytics Dashboards
Lean Analytics & Analytics DashboardsLean Analytics & Analytics Dashboards
Lean Analytics & Analytics Dashboards
 
Combining Methods: Web Analytics and User Research
Combining Methods: Web Analytics and User ResearchCombining Methods: Web Analytics and User Research
Combining Methods: Web Analytics and User Research
 
SAS Institute: Big data and smarter analytics
SAS Institute: Big data and smarter analyticsSAS Institute: Big data and smarter analytics
SAS Institute: Big data and smarter analytics
 
Inventory to Insight to Action with Paula Land
Inventory to Insight to Action with Paula LandInventory to Insight to Action with Paula Land
Inventory to Insight to Action with Paula Land
 
SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offe...
SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offe...SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offe...
SPSNYC15 - An Independent Evaluation of Third-Party SharePoint Analytics Offe...
 
Google Analytics Training - full 2017
Google Analytics Training - full 2017Google Analytics Training - full 2017
Google Analytics Training - full 2017
 
What MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and SurveysWhat MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and Surveys
 
Data Foundation for Analytics Excellence by Tanimura, cathy from Okta
Data Foundation for Analytics Excellence by Tanimura, cathy from OktaData Foundation for Analytics Excellence by Tanimura, cathy from Okta
Data Foundation for Analytics Excellence by Tanimura, cathy from Okta
 
Beyond User Research
Beyond User ResearchBeyond User Research
Beyond User Research
 

Mais de Pronovix

Inclusive, Accessible Tech: Bias-Free Language in Code and Configurations
Inclusive, Accessible Tech: Bias-Free Language in Code and ConfigurationsInclusive, Accessible Tech: Bias-Free Language in Code and Configurations
Inclusive, Accessible Tech: Bias-Free Language in Code and Configurations
Pronovix
 
Creating API documentation for international communities
Creating API documentation for international communitiesCreating API documentation for international communities
Creating API documentation for international communities
Pronovix
 
Docs-as-Code: Evolving the API Documentation Experience
Docs-as-Code: Evolving the API Documentation ExperienceDocs-as-Code: Evolving the API Documentation Experience
Docs-as-Code: Evolving the API Documentation Experience
Pronovix
 

Mais de Pronovix (20)

By the time they're reading the docs, it's already too late
By the time they're reading the docs, it's already too lateBy the time they're reading the docs, it's already too late
By the time they're reading the docs, it's already too late
 
Optimizing Dev Portals with Analytics and Feedback
Optimizing Dev Portals with Analytics and FeedbackOptimizing Dev Portals with Analytics and Feedback
Optimizing Dev Portals with Analytics and Feedback
 
Success metrics when launching your first developer portal
Success metrics when launching your first developer portalSuccess metrics when launching your first developer portal
Success metrics when launching your first developer portal
 
Documentation, APIs & AI
Documentation, APIs & AIDocumentation, APIs & AI
Documentation, APIs & AI
 
Feedback cycles and their role in improving overall developer experiences
Feedback cycles and their role in improving overall developer experiencesFeedback cycles and their role in improving overall developer experiences
Feedback cycles and their role in improving overall developer experiences
 
GraphQL Isn't An Excuse To Stop Writing Docs
GraphQL Isn't An Excuse To Stop Writing DocsGraphQL Isn't An Excuse To Stop Writing Docs
GraphQL Isn't An Excuse To Stop Writing Docs
 
API Documentation For Web3
API Documentation For Web3API Documentation For Web3
API Documentation For Web3
 
Why your API doesn’t solve my problem: A use case-driven API design
Why your API doesn’t solve my problem: A use case-driven API designWhy your API doesn’t solve my problem: A use case-driven API design
Why your API doesn’t solve my problem: A use case-driven API design
 
unREST among the docs
unREST among the docsunREST among the docs
unREST among the docs
 
Developing a best-in-class deprecation policy for your APIs
Developing a best-in-class deprecation policy for your APIsDeveloping a best-in-class deprecation policy for your APIs
Developing a best-in-class deprecation policy for your APIs
 
Annotate, Automate & Educate: Driving generated OpenAPI docs to benefit everyone
Annotate, Automate & Educate: Driving generated OpenAPI docs to benefit everyoneAnnotate, Automate & Educate: Driving generated OpenAPI docs to benefit everyone
Annotate, Automate & Educate: Driving generated OpenAPI docs to benefit everyone
 
What do developers do when it comes to understanding and using APIs?
What do developers do when it comes to understanding and using APIs?What do developers do when it comes to understanding and using APIs?
What do developers do when it comes to understanding and using APIs?
 
Inclusive, Accessible Tech: Bias-Free Language in Code and Configurations
Inclusive, Accessible Tech: Bias-Free Language in Code and ConfigurationsInclusive, Accessible Tech: Bias-Free Language in Code and Configurations
Inclusive, Accessible Tech: Bias-Free Language in Code and Configurations
 
Creating API documentation for international communities
Creating API documentation for international communitiesCreating API documentation for international communities
Creating API documentation for international communities
 
One Developer Portal to Document Them All
One Developer Portal to Document Them AllOne Developer Portal to Document Them All
One Developer Portal to Document Them All
 
Docs-as-Code: Evolving the API Documentation Experience
Docs-as-Code: Evolving the API Documentation ExperienceDocs-as-Code: Evolving the API Documentation Experience
Docs-as-Code: Evolving the API Documentation Experience
 
Developer journey - make it easy for devs to love your product
Developer journey - make it easy for devs to love your productDeveloper journey - make it easy for devs to love your product
Developer journey - make it easy for devs to love your product
 
Complexity is not complicatedness
Complexity is not complicatednessComplexity is not complicatedness
Complexity is not complicatedness
 
How cognitive biases and ranking can foster an ineffective architecture and d...
How cognitive biases and ranking can foster an ineffective architecture and d...How cognitive biases and ranking can foster an ineffective architecture and d...
How cognitive biases and ranking can foster an ineffective architecture and d...
 
APIs: Semi-permeable, osmotic interfaces
APIs: Semi-permeable, osmotic interfacesAPIs: Semi-permeable, osmotic interfaces
APIs: Semi-permeable, osmotic interfaces
 

Último

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 

Último (20)

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 

Making sense of analytics for documentation pages

  • 1. Making sense of analytics for documentation pages Kumar Dhanagopal API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 2. TOPICS What analytics is and isn’t Overview of key metrics The analytics process Interpreting metrics Summary 2 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 3. How analytics can (and can’t) help… ✓ Understand how users engage with our content • Who and from where • How often • On what devices • … ✓ Understand user behavior on our site • How they navigate • What they click • Where they spend time • … • Analytics can’t help us learn about user satisfaction and sentiment • What do users need? • Did they find it? • Were they satisfied? • How (and how much) did they read? • Need other sources (e.g., user ratings, comments) • … 3 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 4. Data != reality (correlation doesn’t imply causation) 4 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 5. The key metrics 5 A B C D E F G H Page Views Visits Entries Exits Bounces A 1 1 0 0 0 B 2 2 0 0 0 C 1 1 1 0 0 D 3 2 0 0 0 E 1 1 1 1 1 F 2 1 1 0 0 G 2 2 0 1 0 H 1 1 0 1 0 Session-1 Session-2 Session-3 Note: ❖ Repeat visits and long visits are treated differently ❖ Cookie’s matter Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 6. The analytics process 1. Define the problem or goal 2. Get the required data • Metrics • Dimensions • Period 3. Prepare the data • Remove noise • Fix inconsistencies • Aggregate data 4. Analyze, explore, visualize 5. Describe, diagnose, prescribe, predict 6 "If you torture the data long enough, it will confess – to anything" – Ronald H. Coase Minimize the garbage-in-garbage-out risk. Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 7. Interpreting metrics: General guidelines • Validate with other data • Avoid cross-product comparisons • Look at trends, not just absolute numbers • Remember: offline and second-hand are not tracked! • Consider the content type, length, format, and structure • Use analytics data as supplementary input for decision making 7 Page Views A 15k B 14k Jan Feb Mar Apr May Jun 3358 2610 3025 2411 2316 1900 0 0 1583 3308 5015 5029 Page A 25k 25k Page B 25K 25k Page C -- 25k Page D -- 25k Chunking affects analytics! Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics Doc 1 Doc 2 50k views 100k
  • 8. Interpreting low page views • Popularity != page views • Look at trends, not just absolute numbers • Consider the “age” of the page • Check whether the page is discoverable • Keep in mind structural changes • Don’t ignore zero-view pages 8 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 9. Interpreting bounce rate • Possibly the most misunderstood metric • High bounce rate is not necessarily a problem • Text-book guidance might not apply to documentation websites • Session time-out = bounce • Analyze in conjunction with other metrics, like time spent on page 9 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 10. Interpreting time spent on page • High time spent: • Was the content useful/interesting? • Difficult to understand? • User left the browser open too long? ☺ • High time spent + high page views • Too much content, or too complex? • Opportunity to improve chunking • High time spent + high bounce rate • Landing pages: cause for concern • Other pages: Opportunity to simplify content • Low time spent + high bounce rates • Reference pages: positive indicator? 10 Time spent is NOT calculated for exit pages! Number of users Total time spent (min) Avg. time spent (min) Action after reading… Group A 50 500 10.0 Exit site Group B 50 250 5.0 Another page in the site Total 100 750 7.5 2.5 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 11. Analytics “targets” Should we have analytics KPIs? Documentation sites != e-commerce portals Consider data availability and decision-making culture 11 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 12. SUMMARY ✓ Data != reality ✓ Analysis requires non-trivial effort ✓ Metrics won’t give us all the answers ✓ Focus on users, business goals ✓ The forest AND the trees matter! 12 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 13. SUGGESTED READING Web Analytics - An Hour A Day, Avinash Kaushik Practical Text Analytics, Steven Struhl An Introduction to Data Science, Jeffrey S. Saltz, Jeffrey Stanton 13 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
  • 14. THANK YOU! https://www.linkedin.com/in/kumardhanagopal 14 Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics