Health Care Stories are Good for You Presented by Sharon Manson Singer, Steve Buist and Jennifer Verma. Canadian Association of Journalists, Annual Meeting, April 28, 2012.

1. Sharon Manson Singer, EvidenceNetwork.ca
Steve Buist, Hamilton Spectator
Jennifer Verma, CHSRF

2.  Introduce EvidenceNetwork.ca
 Talk about the Hierarchy of Evidence
 What makes a good health story? A bad one?
 Top ten questions to ask of health experts
about their research
 Where to go for data
 How to assess the quality of data
 Last word to you the audience

3.  EvidenceNetwork.ca links journalists with
health policy experts to provide access to
credible, evidence-based information.

4.  EvidenceNetwork.ca is a non-partisan, web-
based project funded by the Canadian
 Institutes of Health Research and the
Manitoba Health Research Council to make
the
 latest evidence on controversial health policy
issues available to the media.

5.  The Canadian Health Accord is scheduled for renegotiation
in 2014. Canadians will have to make decisions about
many complex health policy issues, including;
 • Aging population impact
 • Rising drug costs
 • Health care accessibility
 • Private sector financing/delivery
 • User fees
 • Sustainability of the healthcare system
 EvidenceNetwork.ca is committed to working with the
media to build a healthy dialogue around Canadian
healthcare.

8.  Systematic Review and  Strongest evidence –
meta-analysis only as good as the
 Use upper tier studies underlying studies
in a synthesis of
research findings
Type of Study Expected Results

9.  Randomized  Well designed with
experiments sufficient sample size
 High quality source of
 Natural experiments exogenous variation
generating comparison
group
 Well designed pre and
post measures
 Analytical techniques
are appropriate
Type of Study How good is it?

10.  Some control in the  Limited source of
assignment of exogenous variables or
some control of selection
treatment or process
 Well designed pre and
post measures
 Appropriate data with
large sample
 Reasonable approach to
estimating
 Correlational studies counterfactuals
Type of Study How good is it?

11.  Studies without a  Credible case selection
with explicit causal logic
comparison group model
 Quality outcome
measures
 Collect feedback from
participants on quality of
 Participant Satisfaction intervention
 Respected individuals or
organizations with
 Expert Opinions explicit rationale for
opinion
Type of Study How good is it?

12.  Exploratory case  Less credible or
studies explicit case selection
criteria, theory of
change or outcome
measure(s)
Type of Study How good is it?

13.  www.EvidenceNetwork.ca
 Sharon Manson Singer, PhD
 smansonsinger@gmail.com

14. Steve Buist, Investigations Editor
The Hamilton Spectator

15. Luigi Di Bella and the “miracle” cure for cancer
(Pulling for the underdog . . .)

16. Autism and the MMR vaccine
(Science is a lot easier if you just make it up . . .)

17. Multiple sclerosis and “liberation therapy”
(The underdog tale, with a modern-day social media
twist . . .)

18. Climate change and the human touch
(What makes for a balanced story . . .)

20. 1. Who is conducting the science?

21. 1. Who is conducting the science?
2. Who is paying for the research?

22. 1. Who is conducting the science?
2. Who is paying for the research?
3. Who is paying the researcher?

23. 1. Who is conducting the science?
2. Who is paying for the research?
3. Who is paying the researcher?
4. Where are the results being published?

24. 1. Who is conducting the science?
2. Who is paying for the research?
3. Who is paying the researcher?
4. Where are the results being published?
5. What was the population being tested?

25. 6. What was the sample size?

26. 6. What was the sample size?
7. How significant are the results?

27. 6. What was the sample size?
7. How significant are the results?
8. What do other people think, and do those
people have their own conflicts of interest?

28. 9. How do these results fit into the
context of what’s already known?

29. 9. How do these results fit into the context of
what’s already known?
10. Are there opposing viewpoints and how
much weight should those viewpoints be
given?

30. Jenn Verma, Director,
Collaboration for Innovation and
Improvement

31. mythbusters
USING EVIDENCE TO DEBUNK COMMON
MISCONCEPTIONS IN CANADIAN HEALTHCARE
31

32. 32

33. THE LATEST RESEARCH SHOWS THAT
WE REALLY SHOULD DO SOMETHING
WITH ALL THIS RESEARCH
33

34. In a review of World Health Organization (WHO) and
World Bank recommendations on five topics
(contracting, healthcare financing, HHR, tuberculosis
control and tobacco control):
◦ 2/8 publications cited systematic reviews;
◦ 5/14 WHO and 2/7 World Bank recommendations
were consistent with both the direction and nature
of effect claims from systematic reviews.
Hoffman SJ, Lavis JN, Bennett S. 2009. The use of research evidence in two
international organizations’ recommendations about health systems.
Healthcare Policy 9(1): 66-86.
34

35. Effective
Focused on
Population Accessible
Health
Adapted with permission from Health Quality Ontario
Appropriat
ely High-quality Safe
resourced and
performance
health systems
Integrated Equitable
(2011)
Patient-
Efficient
centred
35

36. Select
Provincial Subject Non-
F/P/T International
Health Quality Matter jurisdictional
Jurisdictions Reporting
Councils Experts Organizations
Initiatives
NL Centre for Health Canadian Institute
Canadian Institute National Quality
P/T Ministries of Information for Health
for Health Forum (US)
Health Information
Information
NB Health Council Health Council of Agency for Health
Institute for Clinical Canada Research and Quality
Evaluative Sciences Canadian Patient (US)
Commissaire à la
P/T Health Regions (ON) Safety Institute
santé et au bien- NHS Indicators for
être du Québec Canadian Stroke Canadian Quality Improvement
Network Partnership Against (UK)
Health Quality Canadian Cancer Australian
Ontario Cardiovascular Commission on
Statistics Canada Accreditation
Outcomes Research Quality and Safety in
Canada
MB Institute of Health Care
Team
Patient Safety Fraser Institute Quality and
POWER Study (ON) Ontario Hospital Efficiency in Swedish
Public Health Health Quality Health Care
Association
Agency of Canada Council SK Cardiac Care
Network (ON) Heart and Stroke
Health Quality Foundation OECD
Collaboration for
Council AB Excellence in
Canadian Diabetes
Healthcare Quality
Health Canada Association
BC Patient Safety and Alberta Diabetes RAND
Quality Council Surveillance System Cancer Care Ontario
Adapted with permission from the Health Council of Canada (2011) 36

37. 37

38. Health Indicators provide a Dashboard
for Health and Healthcare
They can let you know that things are
running smoothly.
They can alert you to problems that
may need attention.
38

39. Interpreting Data…
 rising BMI (Body Mass Index) doesn’t explain the root
cause of weight gain.
 In 2009, Canadians received 121 CT scans per 1000
people. There were also 8 MRI units per million
population (vs. 12 MRI units per million as the OECD
average). OECD (2011) reports Canada is “lagging
behind,” but there is no agreed-upon benchmark.
 In 2009, Canada had 2.4 physicians per 1000 population
(vs. 3.1 OECD avg), but…
◦ We have more physicians than ever before – Is this about supply or
distribution and deployment?
◦ We also have more nurses per 1000 people (9.4 in Canada vs. 8.4
OECD avg)
39

40.  Comparing apples-to-apples?
◦ Age standardization
◦ Risk adjustment
 Measuring intangibles
◦ e.g., quality of life
◦ Composite indicators
40

41. 41

42. Lavis J. et al. 2009. SUPPORT Tools for evidence-informed health
Policymaking (STP). Health Research Policy & Systems 7(Suppl 1).
http://www.health-policysystems.com/content/7/S1/I1
42

43. “Numbers can’t ‘talk’ but they can
tell you as much as your human
sources can. But just like with human
sources, you have to ask” (Niles, 2007).
Niles R. 2007. Statistics every writer should know: A simple guide to understanding basic statistics, for
journalists and other writers who might not know math. http://nilesonline.com/stats/
43

44. Support Tools for Policy Making
http://www.chsrf.ca/PublicationsAndResources/Research
Reports/Support_Tools_for_Policy-Making.aspx
Mythbusters
http://www.chsrf.ca/PublicationsAndResources/
Mythbusters.aspx
What If?
http://www.chsrf.ca/Programs/HealthcareFinancingInnov
ationAndTransformation/WhatIf.aspx
CHSRF’s Quality of Healthcare in Canada: A Chartbook (2010)
CIHI’s Making Sense of Health Indicators (2011)
HCC’s A Citizen’s Guide to Health Indicators (2011)
CIHI’s Making Sense of Health Rankings (2008)
OECD’s Health Data 2011: How Does Canada Compare? (2011) 44