1. NEJM.org ReMix Analytics
7.12.2010
7/13/2010 1

2. Agenda
• Customer data
• E-mail de-duping
– Process
– Implications
• Current and post-launch analytical solutions
• NetInsight
– Technology
– Variances in metrics
– Demo
• Post-launch reporting
– Mission Critical KPIs and communication plan
– Monthly reports and ad hoc requests
7/13/2010 Slide - 2

3. Customer Data
• Unique fields
• Tags / tag sets
• Groupings
– Audience segments
– Specialties
• Indicators
7/13/2010 Slide - 3

4. E-mail Address De-Duping: Process
• Identify duplicates
– Merge
– Delete*
• Multiple passes
– Registered users
– Subscribers
– Registered users / subscribers
7/13/2010 Slide - 4

5. E-mail Address De-duping: Impact
• ~200,000 duplicate e-mail addresses
– Merged
– Deleted*
• 10% drop in valid e-mail addresses
• To Atypon:
– ~2,177,000 de-duped records (Day 1)
– ~30,000 “changemms.org” records
7/13/2010 Slide - 5

6. E-mail Address De-duping: Impact
7/13/2010 6

7. E-mail Address De-duping: Impact
• Estimated impact on e-TOC
– Sent (#): -9%
– Opened (#): -2%
– Open rate (%): +7%
7/13/2010 Slide - 7

8. Current Solutions
• NetInsight “On-Premise”
• Views / visits DW
• E-alert DW
• Web databases (WEBDB)
• HighWire
• Akamai
• Premiere
7/13/2010 Slide - 8

9. Post-launch Solutions
• NetInsight “On-Demand”
• MMSDW data warehouse
• Atypon
• Akamai
• Mansell
• >>>PowerPivot
7/13/2010 Slide - 9

10. Current to Post-Launch Solutions
• NetInsight “On-Premise” NetInsight “On-
Demand”
• Views / visits DW NetInsight “On-Demand”
• E-alert DW MMSDW
• Web databases (WEBDB) MMSDW
• HighWire Atypon
• Akamai Akamai
• Premiere Mansell
7/13/2010 Slide - 10

11. Analytics in the NEJM Systems Model
7/13/2010 Slide - 11

12. MMSDW
• Dimensions / facts
– Customers (“parties”)
– Customer attributes
– Subscriptions
– Online product orders
– Online order promotion data
– CME usage
– IMC usage
– Alert opt-ins
– Alert activity
• External data sources linked through PowerPivot
7/13/2010 Slide - 12

13. MMSDW Schema
7/13/2010 Slide - 13

14. NetInsight
7/13/2010 Slide - 14

15. NetInsight Technology
• Logfiles (“On Premise”)
– 48 hours behind
– Contain EVERYTHING
– Intense filtering
• Page and event tags (“On Demand”)
– 12 hours behind (9:00 AM / 10:00 PM)
– Deliberately applied
– No filtering
– Pass visitor data
7/13/2010 Slide - 15

16. Sample Page Tag
7/13/2010 Slide - 16

17. What IS Tagged
• All pages
• All steps in CDFs
• Sign-in layers
• Issue archive
• Global header
• Figures / tables
• Audio / video
• Tabbed articles
• Tools
• IMC, CME, My NEJM
• Concept of “zones”
7/13/2010 Slide - 17

18. What IS NOT Tagged
• Search / Filterbank
• More Topics
• Recently of Note
• Perspective hubs
7/13/2010 Slide - 18

19. Data Passed to NetInsight “On-Demand”
7/13/2010 Slide - 19

20. NetInsight “On-Demand” Demo
• Customer dimensions
• Article dimensions
• Scenario reports
7/13/2010 Slide - 20

21. Anticipated Variances
• Intense validation efforts on current site
• Virtually impossible to quantify for ReMix
– Different tracking technology
• Tagged / not-tagged
– More content
• +151,032 articles
• +85,316 images
• Unlimited “page-events” year 1
7/13/2010 Slide - 21

22. Post-ReMix Reporting
• Overall health of site
– Traffic
– Engagement
– Transaction
7/13/2010 Slide - 22

23. Key Performance Indicators (KPIs)
• Mission Critical KPIs
– Monitored daily
– Reviewed weekly
– Observations communicated weekly
– Launch +45 days
• Existing KPIs will not be updated
• All KPIs will be reassessed after 45 days
7/13/2010 Slide - 23

24. Mission Critical KPIs
• Traffic
– # visitors, visits per visitor, new / returning visitor
ratio, logged-in NEJ / ONJ visits as % of total, # referral
visits from Google, Yahoo!, Bing
• Engagement
– page-events per visit, average visit duration
• Transaction
– Weekly CME exams taken, # conversions / conversion
rate by action, # NEJ / ONJ conversions by referral
page, # account confirmations / confirmation rate
7/13/2010 Slide - 24

25. Additional Monitoring
• Percent of site-wide traffic from e-TOC, Online
First
• Content clicked-through from e-TOC by type
• Home page bounce rate
• % PDF views of full content article views
7/13/2010 Slide - 25

26. Monthly Reports
• All data in current reports will continue
– Institution reports TBD
• Merge / consolidate / enhance
7/13/2010 Slide - 26

27. Ad Hoc Requests
• Anticipate extreme need
• Will keep to a minimum days within first 45
• Analytics Blog: self-initiated analysis
7/13/2010 Slide - 27

28. Summary
• Analytics moving to the “next level”
– Technology
– Process
– Robustness
• Still some open questions
– JW integration
– Usefulness of Atypon reports
7/13/2010 Slide - 28

29. Questions?
7/13/2010 29