Building a Better Message: The 10 Variables That Really Matter (The Tool)
Eileen S. Haag, MEd
Oak Ridge Institute for Science and Education, Oak Ridge, TN
Ms. Haag explores key features of the Message Development Tool that includes a database-driven algorithm
that will provide a visual of predicted average health intention based on the message elements coded by the user,
communication strategy, and message development process. Social media components (including a community
forum) provide users the opportunity to share messages, collect feedback, and “follow” expert users who
consistently provided sound message development advice are also available.
1. Eileen Haag e-Learning Applications Group ManagerHealth Communication and Technical TrainingOak Ridge Institute for Science and Education National Conference on Health Communication, Marketing, and Media August 9-11, 2011 Building a Better Message: The 10 Variables That Really MatterThe Tool Division of Cancer Prevention and Control National Center for Chronic Disease Prevention and Health Promotion
2.
3. Code a draft message against those variables to determine the response level or influence the message may have on the target audience or across target audiences.
4. Change message tactics that may improve an audience’s predicted intentions to comply with a message (based on the tool's algorithm derived from Keller and Lehmann's [2008] model).
5. Defend a message based on a tool generated report with data from the literature about how each variable contributes to message effectiveness for a particular audience. Division of Cancer Prevention and Control
6. BACKGROUND METHODS AND RESULTS CONCLUSIONS IMPLICATIONS FOR PRACTICE Division of Cancer Prevention and Control
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18. The Predicted Intentions Score (to quantitatively assess message value) must be clearly displayed and simply explained.
19. Streamline how preliminary informationfor message development is captured (Health Problem, Target Audience, Barriers, Goal/Objectives, Dissemination Channels, Know/Feel/Do). Division of Cancer Prevention and Control
20.
21. Easy revision of messages (including the ability to re-code and produce a new score).
23. Testers saw how the tool would be very useful for multiple audience comparisons on a message
24. Need a name! Message Development Tool needs a name and branding that gives correct impression of what the tool is and doesDivision of Cancer Prevention and Control
25. CDC's MDT is expected for BETA release in February 2012. Messageworks@orau.gov National Center for Chronic Disease Prevention and Health Promotion Division of Cancer Prevention and Control
Notas do Editor
Theoretical Background and research questions/hypothesis: Developed from an evidence-based and expert-recommended approach (described during the first and second panel presentations), the MDT helps users develop a message by incorporating variables that have been found to enhance effectiveness of health communication. The user can code a draft message against those variables to determine the response level or influence the message may have on the target audience or across target audiences. If the predicted response on target audience intentions to comply with the message (based on the tool's algorithm derived from Keller and Lehmann's [2008] model) is low, the tool will prompt users to make changes in message tactics that may result in an improved response. The tool also generates a report with data from the literature about how each variable contributes to message effectiveness for a particular audience.
Theoretical Background and research questions/hypothesis: Developed from an evidence-based and expert-recommended approach (described during the first and second panel presentations), the MDT helps users develop a message by incorporating variables that have been found to enhance effectiveness of health communication. The user can code a draft message against those variables to determine the response level or influence the message may have on the target audience or across target audiences. If the predicted response on target audience intentions to comply with the message (based on the tool's algorithm derived from Keller and Lehmann's [2008] model) is low, the tool will prompt users to make changes in message tactics that may result in an improved response. The tool also generates a report with data from the literature about how each variable contributes to message effectiveness for a particular audience.
Methods and Results (informing the conceptual analysis)(continued):In user testing, some features were identified as essential to tool usability and usefulness and the importance clear, simply explanations and processes were emphasized. Message "Development" versus "Evaluation/Assessment“Most respondents were confused as to whether the MDT was intended to assist in evaluating or developing health messages.All respondents noted that a clear process for development or evaluation should be described upon first entry into the tool. They suggested that this process be highlighted throughout the tool (i.e., Step 1, Step 2) so that users can assess their progress. Predicted Intentions Score All respondents liked the concept of quantitatively assessing the value of a message for one audience or multiple audiences before costly audience testing. Respondents agreed that the score must be clearly displayed and simply explained. Preliminary Information (Health Problem, Target Audience, Vulnerabilities/Barriers, Goals/Objectives, Dissemination Channels, Know/Feel/Do)Most respondents agreed that the preliminary information questions were important to consider prior to message development. In particular, participants liked simplifying objectives to describe what an audience should "Know, Feel, and Do.“Some did not think it necessary to input preliminary information when assessing a message already developed ("upload a message" option). Some respondents questioned whether asking users to input objectives and then re-state them as "Know, Feel, and Do" was redundant. Similarly, some respondents thought asking users to describe their target audience (on the preliminary information page) and then code for age, gender, race, and regulatory focus was redundant. Emphasizing the Science behind the Tool (Keller and Lehman Model)While the Expert Panel recommended that the tool emphasize its science-based process, most respondents were burdened by reading scientific background information.All expressed that they trusted the scientific basis and validity of the tool, because it was developed by CDC. A few respondents were interested in the scientific background of the tool. They preferred that a description of the Keller and Lehman model be put into a background section or tab so that it is available when they want it, but is not interrupting the core actions of the tool.Most respondents were unaware of the Keller and Lehman model; however, all respondents regarded the model's validation using the popular CDC Verb Campaign as a positive sign of its legitimacy.
Methods and Results (informing the conceptual analysis)(continued):Ensuring the Tool is Quick and Easy to UseMany respondents shared that there are many competing priorities at CDC that make it difficult for employees to devote large amounts of time to new tools (even when they add value). One respondents recalled large numbers of tools that have failed at CDC because no one adopted them. Across the interviews, respondents suggested combining pages, creating straightforward processes, and limiting burdensome language to make the tool as simple and quick as possible to use ("make it like the iPad").A few respondents suggested having "Novice" and "Expert" settings to allow non-first-time users to advance more quickly through the tool. Enhancing the Ability to Readily Revise MessagesAll respondents liked the concept of receiving a score and a bulleted list of message improvements following use of the MDT; however, many expressed that the current wireframes did not emphasize the need to revise a message based on the feedback nor did they structurally support easily revising, re-coding, and receiving a new score. Learning While respondents reacted favorably to the contents of the "Learning Center," most disliked having to view a separate page for clarification on topics within the core of the tool (thus leaving the page they were working on). All respondents suggested embedding learning center content, definitions, examples, and required scientific explanation into each page using pop-up boxes ("fly-overs").Many respondents suggested that all first-time users be directed to a tutorial on the MDT before they can proceed to the Home Page. Respondents envisioned that this tutorial would be visually appealing and explain:the scientific background (briefly);purpose and description of the score; benefits to a health communicator (i.e. testimonials, "why should I use this?);the tool's steps/process;the time commitment for each step; and the contents of the final report.Multiple Audience Comparisons for a Message Some respondents noted the importance of assessing messages for multiple audiences, particularly for large health campaigns. They viewed this capability as a key feature that would influence the usefulness of the MDT for their work.