The New England Region's Public Health Digital Library Project was presented by Elaine Martin, DA, and Karen Dahlen. The project aims to build a digital public health library that will help make information resources, such as full-text journal articles, evidence-based guidelines, and systematic reviews available to public health professionals in all 50 U.S. states.

1. Elaine R. Martin, DA, PI
Karen H. Dahlen, MLS, Project Coordinator
April 28, 2011

2. Aspects of the Study
 Review of the Literature
 Needs Assessment
 Public Health Departments to be Included
 Buy-in from Constituents
 Identification of Resources
 Platform to House Resources (Digital Library)
 Training Related to Resources
 Ongoing Evaluation Tied to Data Model
 Test Existing “Article Delivery” Model
 Review Outcomes
 Relationship of NN/LM, CDC, Other Library Partners

3. Information Needs Assessment
 Behaviors of public health researchers and practitioners.
 Preferences of individuals in the public health workforce
related to digital content.
 Identification of duplicative purchases to improve overall
spend and efficiencies.
 Assess time spent finding information and problems
encountered.
 Implement productivity improvement services and
streamline workflow.
 Learn what roles the public health workforce considers
mission critical.
 Track changes related to content priorities, knowledge
understanding and skill levels over time

4. Project Team
Principal Investigator:
Elaine Martin, Director,
University of Massachusetts Lamar
Soutter Library and Director of the
NER (North East Region) NN/LM
Javier Crespo, Associate Director,
NN/LM NER, University of
Massachusetts
Hathy Simpson, Public Health
Coordinator (NN/LM NER)
University of Massachusetts Medical
School
Karen Dahlen, Project
Coordinator --Licensed
Resources, Consultation &
Training
National Network/Libraries of Medicine
New England Region (NER)
http://nnlm.gov/
→
→

5. Departments of Health
Selected PHDs for this Project
 MidContinental Region NN/LM
 Colorado Department of Public Health & Environment
(CDPHE)
 New England Region NN/LM
 Boston Public Health Commission
 Maine Department of Health
 Massachusetts Department of Public Health
 New Hampshire DHHS
 Rhode Island Department of Health
 Vermont Department of Health
 Connecticut Department of Public Health

6. Information Needs of
Public Health Practitioners
 Broad range of resources needed
 Across the Information Continuum
 Challenges related to information retrieval
 Lack of time
 Lack of knowledge
 Lack of access
 Want access to:
 Full-text journal articles
 News, reports, and conference abstracts
 Automatic notification of new information
 Systematic reviews and evidence-based guidelines
LaPelle NR, Luckmann R, Simpson EH, Martin ER. Identifying strategies to improve access to credible and relevant
information for public health professionals: a qualitative study. BMC Public Health. 2006;6:89.
http://www.biomedcentral.com/1471-2458/6/89

7. Public Health Information Continuum
Research Support for Information
Minimum Maximum
Need access to all levels of the information continuum
LaPelle NR, Luckmann R, Simpson EH, Martin ER. Identifying strategies to improve access to
credible and relevant information for public health professionals: a qualitative study. BMC Public
Health. 2006;6:89. http://www.biomedcentral.com/1471-2458/6/89

8. Template For Improved Access
To Public Health Information
 single portal access with a good search engine;
 automatic alerts regarding newly available information;
 access to best practice information in many areas of interest that
extend beyond biomedical subject matter;
 improved access to systematic reviews, summaries and full-text
articles
 Improved access to the grey literature,
 Better methods for indexing, filtering, and searching
(interoperability);
 Effective ways to archive information; (citation mgt software)
 Improvements in existing systems with which they were already
familiar such as PubMed and others….
LaPelle, NR et al. Identifying strategies to improve access to credible and relevant information for
public health professionals: a qualitative study. BMC Public Health. 2006; 6: 89. Published online
2006 April 5. doi: 10.1186/1471-2458-6-89. PMCID: PMC1456961

9. Background Issues
Related to Information Gathering
Volume 11, Feb 26, 2008 (Outsell, Inc.)
Problems Getting
Information
More….
 Not enough time
 Note enough budget
 Lack of Staff support
 Hard to determine quality
 Not knowing what’s available
 Information too hard to find
 Too much information to sort
 Not being able to compare
across information
alternatives
 Information is too hard to use
and access once found
 Having insufficient training
on how to search and use
information
 Information is not timely/not
updated often enough
 Full text is not available
 Information is not
comprehensive enough
 Insufficient search features
and options

10. Interpreting Scientific Research
Moving it to Practice
 How does one find the best available science to improve health?
 What is an evidence-based resource? (Quality of Evidence Working Group)
 What evidence was considered (quality studies vs opinion)?
 Is all information equal? (All information is not necessarily equal)
 Who participated in gathering/interpreting information?
 It takes 17 years to turn 14% of original research to the benefit of patient
care. (Andrew Balas)
Balas AE, Boren S. Managing Clinical Knowledge for Health Care Improvement. Yearbook
of Medical Informatics 2000. Stuttgart, Germany: Schattauer; 2000

11. Gathering and Analyzing EB Information
Rapid Detection of Epidemics
Detecting an anthrax
epidemic one day earlier
would save $1-7 billion
CDC Presentation by
C. Safron citing:
http://www.cdc.gov/ncidod/eid/vol3no2/kaufman.htm
Kaufmann AF, EID, V3, N2

12. PFH’s Council on Linkages
Core Competencies
 Analytic/Assessment Skills
 Community Dimensions of Practice Skills
 Policy Development/Program Planning Skills
 Basic Public Health Sciences Skills
 Communication Skills
 Financial Planning and Management Skills
 Cultural Competency Skills
 Leadership and Systems Thinking Skills

13. Project Goals
1. Identify and test useful e-databases, e-books, e-journals,
and e-articles in selected public health departments.
2. Understand frequency of use of these resources
(including full-text access to selective licensed e-
resources), who uses the resources, and if such access
could improve the quality of work in selected
departments of health.
3. Evaluate usefulness of these resources in relationship to
related to PHF core competencies and PH essential
services.
4. Introduce technology related to information
management: how to save, store, and reuse information.
5. Determine what resources/costs affordable for Public
Health.

14. Project Objectives
 Promote selected e-journals and two databases that contain
evidence-based public health resources in addition to
PubMed:
 Stat!Ref – compilation of full-text E-books (references connect
directly to PubMed).
 Global Health – contains public health information back to
1912; strong in applied health sciences.
 New Features of PubMed/PubMed Central and introduction to
PHPartners .org.
 NEJM to assess interest in full-text use.
 License EndNote X/4 for multiple users.
 Provide subsidized Article Delivery Service to those who
work in the selected public health departments.
 Test the EBPH website at the Soutter Library.
 Evaluate program for cost and extensibility.

15. Library Partners
• Poudre Valley Hospital System, Ft. Collins, CO
• Maine Medical Center, Portland, ME
• Maine State Library, Augusta, ME (Books)
• Dana Medical Library, University of Vermont,
Burlington, VT
• Rhode Island Hospital/Lifespan/Providence, RI
• William A. Hinton, State Laboratory Institute Medical
Library and Lemuel Shattuck Hospital (MPHD)
• Lamar Soutter Library, University of Massachusetts
Medical School, Worcester, MA

16. Maine CDC Digital Library Page

17. CDPHE—Digital Library

18. Summary of
Processes Related to PLAN
 Hold on-site meetings with Public Health Departments (PHD)
 Identify 3-4 key people within each PHD
 Identify Resources (E)
 Design a Digital Library Template with input from PHDS (E)
 Begin an Environmental Scan to learn about organizations
 Convene a meeting of Project Personnel & PHD lead-team persons (E)
 Select “training team” and develop “standardized curriculum”
 Design training to accommodate local needs
 Train up to 15 people in 7 states (Connecticut will make 8) (E)
 Facilitate/Customize Digital Library intranet access
 Identify Library Partners to test and gather data on “article delivery service”
 Gather and monitor statistics (E)
 Marketing and advertising resources (E)
 What new partnerships could be developed?
 Evaluate , Evaluate, and Evaluate (E)

19. Public Health Commitment
 Contact persons (3): Oversight, IT, Team-Lead to
facilitate training, marketing, and access.
 Onsite visit to introduce the project .
 Identify coordinator and lead-team group onsite.
 Attend “train-the-trainer” session in Worcester.
 Coordinate submittal of surveys and other information
gathering techniques.
 Coordinate on-sight 2 (half-day ) training sessions.
 IT commitment related to “digital library page” install;
 IT EndNote X/4 install and management

20. Increased Access to e-Resources
 Databases (including access to full-text articles)
 PubMed (PubMed Central) – NLM
 Stat!Ref (30+ e-texts) including Red Book
 Global Health (archive + current files)
 E-Tools (Citation Management Software)
 EndNote (to easily store, retrieve, use or reuse literature, pdf’s or
charts)
 Improved Access to Alerts (through databases)
 E-Access to Full-text Journal Articles
 New England Journal of Medicine
 Streamlined Article Delivery (for duration of the project)
 Article Delivery Service through Loansome Doc
 Based on New/Renewed Library Partnerships
 Test of U Mass Soutter Library’s EBPH Website
 List of Journals in Full-Text
 Public Health Journals by Impact Factor

21. How were resources chosen?
 Contain evidence-based, full-text public health content;
 Connect/interoperable with other resources;
 Link to some full-text (the e-article +)
 Vendors were willing to:
 minimize cost to test usefulness of content;
 maximize access;
 provide usage statistics;
 support training; and
 assist with license review and understanding.

22. Use of EndNote
 Save and store bibliographic references for use/reuse:
Manually, through import process, or direct export.
 Organize references, e.g., through “custom groups.”
 Use and understand the “cite as you write” feature
for publication, policy and document preparation.
 Added features, e.g., full text, pictures or graphs to
enhance access or improve document
preparation.
 Ability to store pdf with record for quick information
retrieval.

23. Access to Article Delivery Service
Subsidized Interlibrary Loan
 Access to LoansomeDoc will
be provided by a local library
of first resort.
 Require registration to
Loansome Doc to monitor
usage.
https://docline.gov/loansome/login.cfm
 Maintain established
relationships with existing
libraries within each state.
 Utilize University of
Massachusetts Soutter
Library as last resort.

24. Evaluation
Components
To advance library/information
resource access in Public Health
Departments, there is a need to
know:
What resources are
needed?
Which e-journals are core
to the work?
How information can be
more effectively accessed?
How much will these
resources cost?
 Logic Model
 Resources, activities, outputs,
outcomes
 Evaluation Plan
 Environmental Scan (ongoing
process)
 Journal SurveyMonkey Baseline Data
 Pre/Post Training Surveys
 Metrics (use of resources)
 Just in time “information checks”
 Project End Survey

25. NER PHD New Digital Library
Project Outcomes (3/20/2011)
 New and renewed collaborations were strengthened with 4
hospitals and 3 public PHD libraries to streamline “article
delivery service.”
 A new suite of resources was added for testing: 40 new e-
journals added and one database.
 Early, constant and committed communication.
 Ongoing evaluation has resulted in completed:
 Journal SurveyMonkey results achieved in 5 PHDs.
 Pre-post training survey results in all PHDs.
 Resource usage has been collected and is being reviewed.
 Three sets of “just-in-time information checks” have been
gathered.

26.  Digital libraries are operational in 7 PHDs
 Number of resources increased from5 to 45 in less than a year.
 New Resources were made available onsite through IP addressing.
 NLM provided a unique link for each PHD to validate use.
 18 copies of Endnote were provided to each PHD.
 PR and Marketing Plan now in place
 Seven onsite visits to become familiar with PHD partners and
environment.
 16 training sessions were held within six months.
 Five (5) vendor partnerships went beyond expectations resulting in
greater support.
NER PHD New Digital Library
Project Outcomes (3/20/2011)

27. NER PHD New Digital Library
Project Outcomes
 User Profile
 View Digital Library Pages
 Vendor Splash Pages
 Training Activity
 Sample Journal Survey
Monkey Results
 Unexpected Outcome;
Collaboration w/Vendors
 Use Statistics
 Global Health
 STAT!Ref
 NEJM
 University of Chicago Press
 E-Resources Newsletter

28. Statistical PHD User Profile:
Estimated NER PHD Users of
Trusted Library Resources
Total FTE range from 273
to 4000 for a total of
8468.
We expect about 600 to
be users of these
resources.
Red numbers show
modification to the 10%
rule.
Percentage
based on
Gathered
Data
Potential
Users
PHD Total FTE
30% 38 BPHC* 1100/110
10% 120 CDPHE 1200
20% 79 Maine 395
35% 157 MPHD 4000/1500
15% 40 New
Hampshire
273
10% 40 Rhode
Island
400
15% 45 Vermont 480
10% 40 Connecticu
t
800/400
10% 596 Total ALL 8648

29. Collaboration with Vendors
 Original vendors created splash pages to bring
attention to resources available for this project (sample
follows).
 They supported training components.
 They worked with the separate PHDs (and with the
Project Coordinator to create workable licenses.
 They attended the “Train-the Trainer Session” at the
University of Massachusetts (Worcester) at their own
expense.
 They allowed longer than usual “trials” to stay within
the project framework.

30. Vendors Splash Pages for this Project
http://cabiblog.typepad.com/nerlibrary/
http://www.statref.org/

31. STAT!Ref Usage
July 2010 thru Feb 2011
Retrievals
by Title
Document
Retrievals
Table of
Content
Total
Retrievals
Harrison's Principles of Internal Medicine - 17th Ed. (2008) 2650 2839 5489
Merck Manual of Diagnosis and Therapy, The - 18th Ed. (2006) 1445 1842 3287
Current Medical Diagnosis & Treatment - 49th Ed. (2010) 734 386 1120
Oxford Textbook of Public Health - 5th Ed. (2009) 333 336 669
ACP PIER, Journal Club & AHFS DI Essentials 308 276 584
Public Health & Preventive Medicine - 15th Ed. (2008) 215 209 424
ICD-9-CM - Volumes 1, 2 & 3 (2011) 97 262 359
Red Book: 2009 Report of the Committee on Infectious Diseases - 28th Ed. 111 91 202
INFECTIOUS DISEASES: The Clinician's Guide to Diagnosis, Treatment, and Prevention 73 91 164
ACP Medicine 55 63 118
Casarett and Doull's Toxicology: The Basic Science of Poisons - 7th Ed. (2008) 51 65 116
Evidence Alerts 115 0 115
AHFS Drug Information (2011) 52 62 114
Disaster Nursing and Emergency Preparedness for Chemical, Biological, and Radiological
Terrorism and Other Hazards - 2nd Ed. (2007) 55 37 92

32. Global Health Statistics
Mid June – Mid November
 Global Health Latest News (BBC,
Reuters, Thomson)…reviewing the safety
of GlaxoSmithKline's Pandemrix H1N1
flu vaccine 2/1/2011.
 Global Health has excellent coverage of
public health including journals not in
PubMed, reports, bulletins, conference
proceedings and theses.
 Global Health hosts >11,000 full-text
articles.
 All Global Health foreign-language
abstracts are translated into English.
 Updated weekly
Platf
orm
Jun-
10
Jul-
10
Aug-
10
Sep-
10
Oct-
10
Nov-
10
YTD
Tota
l
CABI
Platf
orm
Tota
l
sear
ches
run
153 6 8 37 193 23 441
CABI
Platf
orm
Tota
l
sessi
ons
32 4 20 14 32 3 117
http://cabiblog.typepad.com/ne
rlibrary/

33. Institution Publication Sep-10 Oct-10 Nov-10 YTD Total YTD HTML YTD PDF
VERMONT
DEPARTMENT OF
HEALTH()
New England
Journal of
Medicine 69 102 118 289 243 46
NEW
HAMPSHIRE
DHHS()
New England
Journal of
Medicine 0 33 61 94 66 28
COLORADO
PUBLIC HEALTH()
New England
Journal of
Medicine 0 4 42 46 33 13
MASSACHUSE
TTS DPH()
New England
Journal of
Medicine 0 0 576 576 469 107
RHODE
ISLAND
DEPARTMENT OF
HEALTH()
New England
Journal of
Medicine 0 0 10 10 6 4
New England Journal of Medicine (NEJM)
Usage September Through December 2010

34. STATE PHD Training Type Date Attendance
Rhode Island RIDH Day 1 9/10/2010 16
Rhode Island RIDH Day 2 10/01/2010 14
Vermont VDPH Day 1 9/21/2010 10
Vermont VDPH Day 2 2/18/2011 13
Maine Maine CDC Day 1 9/27/2010 19
Maine Maine CDC Day 2 9/28/2010 14
Massachusetts BPHC Day 1 10/4/2010 7
Massachusetts BPHC Day 2 10/5/2010 7
Massachusetts MPHD Day 1 11/15/2010 16
Massachusetts MPHD Day 2 11/16/2010 14
New Hampshire NHDHHS Day 1 11/18/2010 13
New Hampshire NHDHHS Day 2 11/19/2010 11
Colorado CDPHE Day 1 01/27/2011 16
Colorado CDPHE Day 2 01/28/2011 14
Total 14 184
Summary of Training Activity

35.  Indentify Name/No of existing journal subscriptions.
 Who pays for subscriptions (individual or institution)?
 What journals would those surveyed like to access?
 Are these journal subscriptions in e- or print format?
 How else do you receive journal articles for public
health work?
 What libraries are used?
 What frequency are other libraries used?
Why a “Journal SurveyMonkey”

36. Unexpected Outcomes (Comments)
Journal SurveyMonkey
 Depends on the topic and lit search
EPE covers chronic disease, injury and violence, and
maternal and child health.
 If work-related journals were available, I’d read them
 I’m not aware of any journals that the Division subscribes to
 We need access to a wide variety of journals
 I would like to have access to the full-range of journal options
that are available to academia
 I am not aware of any journals that our program receives
regularly
 Varies widely depending on specific topic currently under
discussion/exploration.
 Varies widely on the health topic

37. Public Health
Department
No of Surveys Sent Survey Response
Colorado (CDPHE) 254 158 (62%)
Maine CDC 100 76 (76%)
New Hampshire DHHS 65 27 (41%)
Rhode Island 50 19 (38%)
Vermont PHD 480* 72 (15%)
BPHC 15
MPHD
NER PHD Journal SurveyMonkey
Results to Date 3/20/2011

38. Collaboration with EndNote
(Thomson Reuters) Benefits
Output Styles for all Public Health Customers
 ftp://support.isiresearchsoft.com/pub/pc/styles/endnote4/
Morbidity%20Mortality%20WeeklyReports.ens
 ftp://support.isiresearchsoft.com/pub/pc/styles/endnote4/
Pediatrics.ens
 ftp://support.isiresearchsoft.com/pub/pc/styles/endnote4/
Prevent%20Chronic%20Disease.ens
 ftp://support.isiresearchsoft.com/pub/pc/styles/endnote4/
Emerging%20Inf%20Disease.ens

39. EndNote: Adding EID Style
EID: Emerging Infectious
Disease
MMWR: Morbidity &
Mortality Weekly Report

40.  Poudre Valley Hospital System, Ft. Collins, CO.
 Maine Medical Center, Portland, ME
 Maine State Library, Augusta, ME (Books)
 Dana Medical Library (Burlington, VT)
 Rhode Island Hospital/Lifespan/Providence, RI
 William A. Hinton, State Laboratory Institute Medical
Library and Lemuel Shattuck Hospital (MPHD)
 Lamar Soutter Library, University of Massachusetts
 National Library of Medicine NN/LM
Librarian Partners
NER PHD New Digital Library Project

41. Poudre Valley Health System
CDPHE Article Delivery Service
58 CDPHE employees have
registered for Loansome
Doc.
28 Came from the PVHS
Medical Library.
6 Were free online
23 Came from MAUMAS
32 Were filled by other
Resource Libraries
Poudre Valley Hospital
Medical Center of the Rockies
Mountain Crest Behavioral
Healthcare Center
Harmony Campus
Article Delivery Statistics
October 2010 – February 2011
Jerry Carlson, MLS, AHIP
Library Director

42. Marketing the Digital Library Project e-Newsletter
Reliable Public Health e-Resources
E-Newsletter supports
Sending to a Colleague

43. PHD Units/Divisions
Responding to Journal SurveyMonkey
Colorado Dept of Public Health &
Environment (CDPHE) New Hampshire DHHS
What Division are you in?
Answer Options
Response
Percent
Response Count
Administrative and Financial
Services Division
1.3% 2
Executive Director's Office
3.8% 6
DCEED
25.3% 40
CHEIS
3.8% 6
PSD
31.6% 50
HFEMS
8.2% 13
EPRD
5.7% 9
HMWMD
5.1% 8
APCD
7.0% 11
WQCD
1.9% 3
Laboratory Services
1.3% 2
Environmental Health and
Sustainability
3.8% 6
Other Environmental Programs or
Divisions
1.3% 2
Answer Options Response Percent Response Count
Public Health Systems, Policy &
Performance
7.4% 2
Public Health Laboratories
18.5% 5
Infectious Disease Control
11.1% 3
Population Health & Community
Services
29.6% 8
Public Health Protection
18.5% 5
Health Services Planning and Review
3.7% 1
Public Health Statistics and Informatics
7.4% 2
Director's Office
3.7% 1

44. Marketing Techniques

45. Evaluation Methods
Ongoing in NER PHD Project
 Information Collected from Lead-Team Persons Related to
Type of Information Needed.
 These 8 topical areas became the essence for Pre/Post
Training Surveys
 Journal SurveyMonkey (tested in Maine)
 Only one PHD has not completed
 Review of Resource Licenses
 Pre/Post Training Surveys
 Customized links for both Surveys
 Review of “article requests” from Library Partners
 Project End Survey

46.  What Role Can the CDC Play in this Project?
 How Can the CDC Participate in the National Virtual
PH Library Project?
 What CDC Funding Sources Are Available to Expand
this Project?
Questions for Discussion