Overview of three iterations of search analytics tools and actions taken to improve search UX based on analytics.

1. What to Feed Your Search Engine:
The Evolution of Search Analytics at HBS
Ravi Mynampaty
Copyright © President & Fellows of Harvard College

2. Our Organization
• Harvard University
• Many schools (HMS, HLS, HKS, …)
 Harvard Business School (HBS)
 ITG, DRFD, MBA, Exec Ed, M&C, …
 Knowledge & Library Services (KLS)
 Information Management Group
 Metadata/Taxonomy
 Information Lifecycle Management
 Analytics
 Search & Findability (== Ravi)
@ravimynampaty 2

3. Agenda
• Background
• Three generations of analytics tools
• Query Log Reports
• JeffTrends
• Hosted solution
• Using analytics
@ravimynampaty 3

4. Background
• In the beginning…
• OOTB search engine
• No optimization, no customization
• Fraction of HBS content indexed / searchable
• Proliferation of different search tools
• User sentiment
 “Search sucks”
 “Why can‟t we just get Google?”
@ravimynampaty 4

5. @ravimynampaty 5

6. Query refinement options
Query resubmit options
@ravimynampaty 6

7. Analytics Needed…
• Provide insight into:
 Impact of UI changes
 Behavior of search users
 needs, wants, give a „face‟ to visitors
• Reflective, not predictive
@ravimynampaty 7

8. Common Metrics to Track
• Visitor Segmentation
• Visits
• Top Entry/Exit Pages
• Task Completion Rates
• Visitor Primary Purpose
• Query Analysis
@ravimynampaty 8

9. Top search terms
• Do they have good results?
• Any with no-hits?
• What‟s the click-thru rate ?
• What are the referrer pages ?
@ravimynampaty 9

10. Analytics: Implementation
• Query Log Reports
• Record additional data as needed
• Refinement usage
• Best bets click-thru
• Originating site
• Generate reports from logs
• JeffTrends
• Collect data with JavaScript
• Record to web server log
• Generate reports from logs
@ravimynampaty 10

11. Query Log Reports
@ravimynampaty 11

12. Query Log Reports
@ravimynampaty 12

13. Query Log Reports
@ravimynampaty 13

14. JeffTrends
@ravimynampaty 14

15. JeffTrends
@ravimynampaty 15

16. Top Queries by Referrer
@ravimynampaty 16

17. Top HBS Queries by Role and Profile
@ravimynampaty 17

18. Path from www
@ravimynampaty 18

19. Using Analytics: The Process
• Collect Data
• Record data (reports)
• Test improvement strategies
• Implement improvement
• Measure results
• Repeat process
@ravimynampaty 19

20. Using Search Analytics
• Add/remove refinement options
• Add synonym-like functionality
• 5-2423  (617) 495-2423
• Date sorts
• Taxonomy development
• Feed Best Bets
@ravimynampaty 20

21. Create Best Bets
Top 10 Queries
Oct – Dec
@ravimynampaty 21

22. Conclusion
• Getting the most out of your search engine
• Search is only as good as content
• Content gaps/content development/design
• Answer “What if …?” questions
@ravimynampaty 22

23. Thank You
• Questions?
searchguy@hbs.edu
@ravimynampaty 23