The McGovern Award is given by the American Medical Writers Association for "preeminent contributions to medical communications." It was presented at the AMWA annual conference, in Memphis, October 9, 2014.
McGovern Award Lecture - American Medical Writers Association
1. Too Much Of The
Wrong Kind
Of Health News
Gary Schwitzer
Publisher, HealthNewsReview.org
Adjunct Assoc. Prof., School of Public Health, University of Minnesota
2.
3.
4.
5. Our criteria: Does the story
explain…
• What’s the total cost?
• How often do benefits occur?
• How often do harms occur?
• How strong is the evidence?
• Are there alternative choices?
• Is the condition exaggerated?
• Is this really a new approach?
• Is it available?
• Who’s promoting this?
• Do they have a financial conflict of interest?
69%
66%
65%
61%
57%
Percent
unsatisfactory
after 1,889
story reviews –
7 years
6.
7. Most common flaws
• Conveying a certainty that doesn’t exist
– Exaggerating effect size
– Using causal language to describe observational
studies
– Failing to explain limitations of surrogate
markers/endpoints
– Single source stories with no independent
perspective
– Failing to independently analyze quality of evidence
8. “Good journalism has a subtle feature of
reticence. We don’t publish everything we hear.
We filter. We curate. The goal of the traditional
journalist is to create a reputation for accuracy,
fairness, relevance and timeliness, and this
requires the willingness to not publish things
that are unlikely to be true. … There’s nothing at
stake here except the survival of credible
journalism.”
- Joel Achenbach
Washington Post
21. “Boston Children’s Hospital
could be on the verge of
curing type 1 diabetes.
Seriously. This huge news,
which was announced today
on their blog.”
22.
23. JOSH BILLINGS (PEN NAME OF HUMORIST
HENRY WHEELER SHAW, 1818 – 1885)
“The trouble with people
is not that they don't know, but
that they know so much that ain't so.
…
I honestly believe it is better to know
nothing that to know what ain’t so.”
25. September 2012
Positive “spin” was identified in about half of press releases and
news stories. The main factor associated with “spin” in press
releases was the presence of “spin” in the journal article abstract
conclusion.
In other words, a direct link from
published study news release news story.
Who thinks about the reader, the consumer,
the patient…at the end of this food chain?
26. Mayo’s Dr. Victor Montori to a journal club:
“Beware spin: composite endpoints, surrogate
markers, subgroup analyses, inadequate
comparators (too much or too little of an effective
alternative or placebo when an effective alternative
exists). Above all, avoid the intro/discussion
sections which is where most of the interpretational
spin is introduced.”
But many journalists – who feed off a steady diet
of journal studies in order to meet their story
quotas and click rates – have no idea what any of
these terms mean.
27. Dr. Richard Lehman has published reviews of
journal articles for 8 years on BMJ website
“I too was once a conclusion-of-the-abstract
reader, and was quite smug that I had even got
that far. It took me some years to become
aware of perhaps the most important principle
of critical reading: never believe the stated
bottom line without confirming it from the data.
And beware of the limitations of the data.”
28. Dr. Lehman reminds us:
350 years ago the Royal Society of London for Improving
Natural Knowledge used as its motto,
Nullius in verba
Rough translation:
“Don’t take anyone’s word for it.”
We still have much to learn from those notable 17th
century skeptics.
30. Pharma says that for every 5,000 compounds in pre-clinical
testing, only 5 make it to human trials. And of those, only 1
will get FDA approval.
But we keep seeing stories like:
– The Toronto Star: “New breast cancer drug heralded as breakthrou
(Phase 2 study)
– CNN: “Stem cell medical breakthrough?” (Phase 1 study)
– Medscape: ‘Truly Remarkable’ Response with Combination for
Melanoma (Phase I study – 37 people followed 12 weeks)
Another estimate: only 8% of new drugs in Phase I studies ever get
approved
31. “BREAKTHROUGH” on all 3 main TV
networks on a Phase I study
Note that all 3 practice
“question mark
journalism” – which
allows you to say
anything, followed by ?
33. Stenography, not journalism
Reported on Phase 2 breast cancer drug study – directly from a Pfizer news release –
which didn’t include any data.
But the story lifted a glowing quote from the company news release:
“The study suggests ‘the potential to transform the standard of care,’ said Mace
Rothenberg, Pfizer Oncology's chief medical officer. ‘This is encouraging information
for these women.’ “
No data. No independent perspective. Free publicity on results no one will see for two
more months when they are presented at a meeting.
But Reuters and AP also reported on the Pfizer news release. Those stories were picked
up by Huffington Post, ABC, Charlotte Observer, Boston Herald, Idaho Statesman,
Washington Times, Bradenton Herald, Fort Worth Star Telegram, Myrtle Beach Sun News,
Philly.com, others
34. Why does any of this matter?
• Because we are often mired in an ugly,
uninformed public dialogue about health care
35. Message from a researcher who promotes
mammography benefit
to a researcher whose work questions benefit
“May the faces of the thousands of women who
have died, and will continue to die, prematurely
because of you, continue to haunt you in your
dreams, and in your nightmares.”
36. The management of
• The CEO said the US Preventive Services Task
Force “condemns tens of thousands of men to
die this year and every year going forward…”
• The COO said that Dr. Otis Brawley, chief
medical officer of the American Cancer
Society “has killed more men by giving them
an excuse to not be tested.”
37. When an FDA advisory committee in
2011 recommended against Avastin
for breast cancer
The VP of the National Breast Cancer Coalition - a breast
cancer survivor - told the panel the data don’t show that
the drug extends the lives of breast cancer patients, but
do show it increases risk of harm, especially risk of
hemorrhages.
• Next, a woman got up to give her public presentation
and started by saying, “I am disgusted to have to speak
after that woman.” Her statement was met with
applause.
• One woman yelled, “What a patient representative!
You better hope your breast cancer doesn’t come back.
You’re an embarrassment to all cancer survivors. ”
38. Respected patient advocate Musa Mayer said:
“As a patient rep for the FDA’s Oncologic Drugs
Advisory Committee, I, too have been publicly
reviled for making evidence-based decisions
that I felt would benefit the larger population of
patients. Some of us have had to grow
accustomed to receiving hate mail, hurled
epithets intended to shame us, and even threats
from those who claim to speak on behalf of
patients.”
39. Journalism has an agenda-setting
responsibility on issues like this
But the story of the forces behind this wretched
public dialogue on screening and on evidence-
based medicine is largely untold.
• Are the lobbying forces of special interests so
powerful?
• Are journalists afraid, unaware, don’t consider it
newsworthy?
• Any one of these possibilities is unacceptable.
40. Journalists could help people
understand
• In health care, newer isn’t always
better…more isn’t always better…less can be
more.
• Bad things happen when we re-define normal
states of health as illness requiring treatment
• There are tradeoffs involved in any health care
decision: something you stand to gain but also
something you stand to lose.
41.
42. St. Augustine & Jerry Garcia
“Hope has two beautiful
daughters. Their names are
anger and courage; anger at
the way things are, and
courage to see that they do
not remain the way they are.”
“Somebody has to do something,
and it's just incredibly pathetic
that it has to be us.”
Notas do Editor
It’s October. Baseball World Series Time. This man, Yogi Berra, appeared in a record 21 World Series, winning 13 of them as a player, coach, or manager,
It’s October. Baseball World Series Time. This man, Yogi Berra, appeared in a record 21 World Series, winning 13 of them as a player, coach, or manager,
Yogi is credited with the line: “If you come to a fork in the road, take it.” That kind of map doesn’t help.
But it appears that many of the people who write about health care follow Yogi’s advice.
Each day they come in to work, they face a fork in the road in decision-making about which stories to write and how to write them. And they forge ahead. They write as if there is certainty. As if all the evidence is in. As if we know that the newer, bigger, costlier intervention is better.
But – almost always – we don’t know that.
There was a fork in that road.
And the road not taken is the path of uncertainty. The trail of evidence. The avenue of independent scrutiny, not simply stenography.
Readers – news consumers, who are also health care consumers – need a map for that road not taken. They need journalists and other communicators who help them navigate the terrible confusion of a complicated health care system and all of its offerings of treatments, tests, products and procedures.
What they don’t need are news stories and other media messages that suggest we know what we’re doing – we know where we’re going – when often we don’t.
These aren’t off-the-cuff observations.
For 8 years, I’ve published HealthNewsReview.org, a project that tries to improve the public dialogue about health care by analyzing media messages. The team I led conducted systematic reviews of nearly 2,000 news stories by leading organizations…applying these 10 criteria.
The report card is not good.
60-70% of those nearly 2,000 stories were judged unsatisfactory on what are arguably the five most important of our 10 criteria. For the information consumers need.
That means 60-70% of our huge sample was not ready for prime time. These stories made most interventions sound terrific…risk-free…and without a price tag….and to hell with the evidence and to hell with meaningful comparisons with existing alternatives.
This work was summarized this summer in a paper I had published in JAMA Internal Medicine.
The most common flaws, summarized in that paper, were news stories that:
Conveyed a certainty that doesn’t exist
Exaggeraed effect size
Used causal language to describe observational studies
Failed to explain limitations of surrogate markers/endpoints
Single source stories with no independent perspective
Failed to independently analyze quality of evidence
Let me talk squarely out of both sides of my mouth.
We are seeing terrific peaks of excellence in health care journalism – usually in-depth, investigative, data-driven pieces – often foundation-funded.
We have too many writers who treat too many studies published in too many obscure journals as if they were etched in stone and coming down the mountaintop with Moses. But feeding off a steady diet of papers in journals is an unhealthy diet – for journalists – and for the news consumers who read their stuff.