Measuring Health Literacy: Assessing Current Approaches Dr. Andrew Pleasant

1. Measuring health literacy:
Assessing Current Approaches
Andrew Pleasant, Ph.D.
Health Literacy and Research Director
Canyon Ranch Institute
Northern New Jersey Maternal / Child Health
Consortium
June 5, 2012
Bergen, NJ

2. Disclosures
The following speaker has financial relationships with
commercial interests to disclose:
• Andrew Pleasant, Ph.D.
• Is a member of the Merck Speaker’s Bureau on health
literacy.
• Time to Talk CARDIO is an educational program paid for by
Merck and developed in partnership with the American
Academy of Family Physicians Foundation, Canyon Ranch
Institute and RIASWorks.

3. Summary slide: The current state
of health literacy measurement

4. Who here …
• Has used any of the existing
measures or screeners of
health literacy?
• How did that go?

5. Existing screeners and measures - 1
• Rapid Estimate of Adult Literacy in Medicine
(REALM) (Davis et al., 1991; Davis et al., 2006; Davis
TC, 1993)
• Test of Functional Health Literacy in Adults
(TOFHLA) (Gong et al., 2007; Parker, Baker, Williams,
& Nurss, 1995),
• Health Activities Literacy Scale (HALS) (Rudd,
Kirsch, & Yamamoto, 2004),
• Newest Vital Sign (NVS) (Weiss et al., 2005),
• Wide Range Achievement Test Fourth Edition (Dell,
Harrold, & Dell, 2008),

6. Existing screeners and measures - 2
• Stieglitz Informal Reading Assessment of Cancer
Text (SIRACT) (Agre, et al., 2006),
• Medical Achievement Reading Test (MART)
(Hanson-Divers, 1997),
• National Adult Reading Test (NART) (Uttl, 2002),
• Literacy Assessment for Diabetes (LAD) (Nath,
Sylvester, Yasek, & Gunel, 2001),
• Nutrition Literacy Scale (NLS) (Diamond, 2007),
• Short Assessment of Health Literacy for
Spanish-speaking Adults (SAHLSA) (Lee,
Bender, Ruiz, & Cho, 2006),

7. Existing screeners and measures - 3
• an instrument targeting Canadian adolescents,
• a “talking touchscreen” approach,
• Demographic Assessment of Health Literacy
(DAHL) (Hanchate, Ash, Gazmararian, Wolf, &
Paasche-Orlow, 2008).
• Items from the 2003 National Assessment of
Adult Literacy (Baldi et al., 2009)
• Health Literacy Skills Instrument (HLSI)
(McCormack et al., 2010)
• Mandarin Health Literacy Scale (MHLS) (Tsai,
Lee, Tsai, & Kuo, 2011)

8. Existing screeners and measures - 4
• The Agency for Healthcare Research and
Quality (AHRQ) developed a “health literacy
item set” for the Consumer Assessment
of Healthcare Providers and Systems
surveys ((AHRQ), 2007).
• The Joint Commission is embarking on an
effort to develop health literacy standards
as part of its hospital accreditation process.
• The Eurobarometer has recently completed a
health literacy assessment in eight countries.

9. Most new measures are validated
against older measures – but …
Measure Exact description General description
N=207; convenience sample; 54% black; 76% Black women with
REALM
female; 42% dropped out of high school less education
N=403; app. 20% refusal; 11% failed screening; Hispanic and
convenience sample, 45% African American African Americans
TOFHLA
“indigent”; 45% Hispanic; 58.5% less than with less
high school graduate/GED. education
N=500 (250 eng; 250 Spanish); 20% refusal;
Newest
mean age 41; 21.5% white, 73% Hispanic; Hispanic women
Vital Sign
84 men; 416 women
Chew’s N=332; 5% women; 81% white; 86% GED or
single higher; ambulatory pre-op clinic (excluded White men with
item ‘worst’ cases) GED or higher
screener
Wallace’s N=305; 68% female; 81.3% insured by
White women with
single TennCare/Medicare; only English speaking;
less than high
item 85.2% White; 88% less than high school
school education
screener education

10. The
REALM
Impetigo

11. Let’s take (part of) the TOFHLA!
• Cloze method - multiple choice
• Fyi, this isn’t the only design for the cloze method. Imagine
the difference if there were NO choices (exact answer/
acceptable answer) .. Or you struck (X-ed) out an incorrect
word and replaced it… or, score by difficulty of word? Etc.

12. TOFHLA: An example

13. Troubles with the TOFHLA: A brief example
• Average refusal + exclusion = 40% (n=48)
• No consistency in use or reporting:
○ Meta-analysis of data is NOT possible
• No random samples of the general population
• Meta-analysis population significantly different
than U.S.
• Both ceiling and floor effects
• Several biases identified by researchers
• Inconsistent data - linear or categorical

14. Newest Vital Sign:
A pint of ice cream?
• What is health literacy?
• What is the most important
question in this area of
healthy behavior?

15. Chew / Wallace… final choice
• How often do you have problems learning
about your medical condition because of
difficulty understanding written information?
(note .. 15th – 17th grade level)
• How confident are you filling out medical
forms by yourself? (**Wallace) (10-12th)
• How often do you have someone help you
read hospital materials? (** Chew) (8-10th)

16. A fundamental distinction
• The goal of screening …
○ divide people into healthy and sick categories (have/ have not).
○ In clinical contexts, this demands short, quick & easy to use
• The goals of measurement …
○ advance knowledge - i.e. test hypotheses
○ explore and explain structure and function
○ monitor effectiveness and equity of interventions
○ indicate major problems confronting society
○ contribute to setting policy goals
Equivalent to the difference between an “old-fashioned blood
pressure cuff, stethoscopic, and manual abdominal health
check-up and a comprehensive health examination” (Breslow, 2006)

17. Concerns about screening
• SHAME … a silent barrier
• Almost 40% of patients with low health
literacy who also acknowledged they have
trouble reading admitted shame.
• Of those …
○ 67.2% had never told their spouses
○ 53.4% had never told their children
• Nineteen percent (19%) had never
disclosed their difficulty reading to
anyone.

18. AMA Foundation says …
• Screening/measurement is
fine for research, but it's
not appropriate for daily
clinical practice.
• "Clinicians can better
spend their time ensuring
that all their patients
understand the medical
information they need to
know to care for
themselves."

19. Measuring what?
Health Care Patients /
System Public
Ability to
Level of
Health participate
demand
Literacy (Reception
(Sending skills)
Skills)
Nothing
Some measures skills NAAL and
readability tools; on either side in many
new OSCEs context screeners

20. Critiques of existing screeners: A summary
Existing measures/screeners of health literacy:
• are not designed to test or advance an underpinning
theory of health literacy,
• are limited in approach to evaluating skills - not
behavior change or capacity to change (e.g., some
overly rely on the cloze formatted reading test while
others only evaluate word recognition and not
understanding),
• lack cultural sensitivity and can exhibit bias toward
certain population groups,
• do not measure an individual’s ability to prevent
illness and injury,

21. Critiques of existing screeners: A summary
• are not directly useful for informing or
evaluating health promotion and
communication interventions (e.g., a pre-post
design), curricula, policy, or schemes to pay
physicians based on performance,
• place a problematic burden and potentially
harmful label on patients being tested in
clinical settings,
• do not evaluate spoken communication skills,
• do not consider health literacy as a public
health issue,

22. Critiques of existing screeners: A summary
• ambiguous item wording,
• do not adequately distinguish between people at
very low and very high levels of health literacy,
• were not subjected to rigorous psychometric
analysis,
• have not been used in a consistent way,
• focus on a single dimension while health literacy
involves multiple dimensions,
• may be biased toward those with recent experience
with the health care system or content area, and

23. Critiques of existing screeners: A summary
• the variations among the tools and how they
have been used make it difficult to compare
experiences or results across studies to
definitively establish the relationship of health
literacy to health status.

24. The NIFL (now LINCs) discussion:
Deciding what’s needed
• Over 200 messages
• Approximately 80 contributors
• At conclusion of week long discussion:
○ Created an online survey tool
○ 123 respondents
○ 4 day time frame

25. Possible consensus statements and responses
Strongly Strongly
Disagree Disagree Agree Agree
New measures of health literacy need to be
1% 8% 38% 53%
developed.
New measures of health literacy need to be 0% 5% 44% 51%
based on sound theory.
Measurement of health literacy needs to be 0% 2% 38% 60%
relevant to actual experiences.
Existing measures of health literacy, while
important to the early development of the
2% 11% 44% 44%
field, do not match the understanding of
health literacy that has developed.

26. Possible consensus statements and responses
Strongly Strongly
Disagree Disagree Agree Agree
We need to be able to measure both sides of
the health literacy equation - the health
1% 2% 27% 70%
literacy of individuals and the health literacy
of systems/health professionals.
Health literacy measurement should not be
4% 21% 38% 38%
prioritized in the clinical context.
No single methodological tool is up to the
task of measuring health literacy, therefore a
measure of health literacy must incorporate
1% 8% 44% 48%
multiple methodologies. This may include
both quantitative and qualitative
methodologies.

27. Possible consensus statements and responses
Strongly Strongly
Disagree Disagree Agree Agree
A measure of health literacy needs to be
validated with a broad population, not just a 0% 4% 35% 61%
limited sample.
A measure of health literacy should include
0% 20% 48% 32%
evaluation of spoken language skills.
A measure of health literacy will be multi-
dimensional, addressing both multiple 1% 4% 44% 51%
conceptual domains and multiple skills.

28. Possible consensus statements and responses
Strongly Strongly
Disagree Disagree Agree Agree
As you cannot ‘see’ health literacy,
the measure must sample from all
the conceptual domains outlined by
the underlying theory or conceptual 1% 8% 68% 24%
framework. The measure can be
comprehensive but does not have to
include everything.
A measure of health literacy that
focuses solely on the clinical setting
is inappropriate when researching 2% 11% 43% 45%
public health behaviors and
outcomes.

29. What should be included in a measure of health literacy?
Strongly Strongly
Disagree Agree
Disagree Agree
Finding/obtaining 0% 7% 43% 51%
Understanding 0% 0% 22% 78%
Evaluating/processing 0% 2% 34% 64%
Communicating/ Being able to
0% 0% 35% 65%
communicate
Using information 1% 2% 28% 69%
Making informed choices 1% 5% 26% 68%
Making appropriate choices 7% 17% 28% 49%

30. How should you build a new measure?
• Explicitly built on a testable theory or conceptual
framework of health literacy.
• Multidimensional in content and methodology.
• Measure on a continual, not a categorical basis.
• Treat health literacy as a ‘‘latent construct.’’
• Honor the principle of compatibility.
○ E.g. basing measurement of health literacy on an ice
cream nutritional label is not compatible with a
clinical setting.

31. How should you build a new measure?
• Allow comparison to be commensurate across
contexts including population groups, cultures,
life courses, health topics, and research
settings.
• Prioritize social research and public health
applications versus clinical screening.
• Others you may suggest?

32. What I suggest you do now!
• Treat health literacy as the way to shape the
intervention – measure desired outcomes.
• Use the Calgary Charter on Health literacy
model to guide the design of your
intervention.
FIND EVALUATE
COMMUNICATE USE
UNDERSTAND

33. • Now it is your turn!
What do you think should come next
for health literacy measurement?
What are you doing now?

34. Thank you!
Andrew Pleasant
andrew@canyonranchinstitute.org