Hypothesis Tested: Designing Games with Theoretical Frameworks
•Transferir como PPTX, PDF•
1 gostou•128 visualizações
With games being designed that seek changes in intention, attitudes and behavior it’s more important than ever that we evaluate the games we design and understand their true effects. A team from the University of Miami designed a game-based intervention with the goal to reduce intentions to indoor tan. We found that games can be a powerful persuasion tactic but not always in the ways one would expect! Join us as we share our design process and findings, which indicate the importance of knowing your target audience and how there isn’t a one-size fits all approach to designing such interventions.
Recomendados
Mais conteúdo relacionado
Semelhante a Hypothesis Tested: Designing Games with Theoretical Frameworks
Semelhante a Hypothesis Tested: Designing Games with Theoretical Frameworks (20)
Mais de Clay Ewing
Mais de Clay Ewing (9)
Último
Último (20)
Hypothesis Tested: Designing Games with Theoretical Frameworks
- 1. HYPOTHESIS TESTED Designing Social Impact Games with Theoretical Frameworks Clay Ewing @claytical c.ewing@miami.edu Associate Professor University of Miami
- 3. HUSTLING HEALTH CARE (2010) A tabletop game that simulated health insurance markets before the Affordable Care Act.
- 4. TANZANIAN SOCIAL ACTION FUND (2012) Working with the Tanzanian government and the World Bank, we developed a facilitated tabletop game intended to teach rural farmers about a new conditional cash transfer program.
- 5. “ ” IF YOU CAN MEASURE IT, THEN IT’S NOT THE CHANGE I WANT TO SEE… IF YOUR GAME OR TECHNOLOGY REALLY ‘WORKS’ IT FREAKS ME OUT Paolo Pedercini, Making Games in a F****d Up World , Games for Change 2014 Pedercini, P. (2014) Making Games in a F****d Up World, Games for Change 2014. Retrieved from https://www.youtube.com/watch?v=MflkwKt7tl4
- 6. VANITY A tabletop game where players take on the role of aspiring actors competing at auditions. The game aims to teach teenagers about the dangers of using tanning beds. Preliminary results are inconclusive but promising. The game may be an acceptable preventative tool to keep people from engaging in tanning behavior that could lead to skin cancer. Tran, Carcioppolo, Colantonio, Ewing, Lang, Beyea, Beecker. “The Empty Comfort of Vanity: Assessing the Effectiveness of an Interactive Game to Increase Skin Cancer Prevention Outcomes” (GLS10)
- 7. UNSAVORY (2015) You play as a minimum wage worker in a fast casual restaurant preparing burritos. Your goal is to survive a month on the job. Eventually, your work is interrupted by an oncoming sneeze. If you sneeze, your boss knows you’re sick and sends you home. If you miss too much work, you’re fired and won’t be able to pay your bills.
- 8. “ ” AFTER I ANALYZED THE RESULTS FROM THIS STUDY, I WAS DISMAYED TO FIND THAT PLAYING THE GAME HAD NO EFFECT ON POSITIVE FEELINGS TOWARD THE POOR. IN FACT, THE GAME HAD A NEGATIVE EFFECT ON ATTITUDES AMONG CERTAIN PARTICIPANTS—INCLUDING SOME PEOPLE WHO WERE SYMPATHETIC TO THE POOR TO BEGIN WITH. Gina Roussos, When Good Intentions Go Awry, Psychology Today Roussos, G., & Dovidio, J. F. (2016). Playing below the poverty line: Investigating an online game as a way to reduce prejudice toward the poor. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 10(2). Roussos, G. (2015). When Good Intentions Go Awry: The Counterintuitive effects of a prosocial online game. Psychology Today. Retrieved from https://www.psychologytoday.com/us/blog/sound-science-sound- policy/201512/when-good-intentions-go-awry
- 9. “ ” WHEN IMPACT IS DEFINED TOO NARROWLY, SOME GAMES ARE DISMISSED FOR THE WRONG REASONS AND THEIR IMPACT IS OVERLOOKED. Impact with Games: A Fragmented Field Stokes, B., O’Shea, G., Walden, N., Nasso, F., Mariutto, G., Hill, A., & Burak, A. (2016). IMPACT WITH GAMES: A FRAGMENTED FIELD. Pittsburgh, PA: ETC Press. Retrieved from http://gameimpact.net/reports/fragmented- field/
- 10. Integrated Model of Behavioral Prediction: Fishbein, 2000, 2007; Fishbein & Yzer, 2003 Health Belief Model: Rosenstock, 1974; Rosenstock, Strecher & Becker, 1988
- 11. POTENTIAL SUITORS Anti Tanning Neutral Pro Tanning For each tanning belief, we created 3 personality types that would cover different combinations of tanning beliefs and life values.
- 12. CONVERSATIONS
- 13. IN GAME TANNING OFFERS Tanning Salon Dermatologist
- 14. THE EPILOGUE The final piece of the game has an anti- tanning health message implicitly presented. It gets shown when the character went to a tanning salon or they chose to date someone with a history of using tanning beds.
- 15. STUDY RESULTS The game was the most effective intervention at reducing intent to tan with women and people who haven’t indoor tanned previously. However, they did not enjoy the game as much as the other interventions. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 16. STUDY RESULTS There was a boomerang effect! Men and people who have indoor tanned previously enjoyed the game and are more likely to tan after playing the game. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 17. STUDY RESULTS We assumed increasing engagement would have a positive effect but the increase in variables can decrease the persuasive effects of the game. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 18. STUDY RESULTS In traditional interventions, increased transportation leads to better outcomes. In the game condition, the players were transported but it had a negative effect. We can’t make a conclusion on this yet. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 19. It would be irresponsible to publish the game knowing what we found during the first study
- 20. When we figure out how to eliminate the boomerang effect, the game will be a solid health intervention tool.
- 21. THE ROAD AHEAD • Limited character versions that focus on one category (tanning vs. anti-tanning) • Deep dive into specific character messages and player behaviors
- 22. DREAMY – NON TANNING EDITION The idea of interacting with a series of characters with the pretext of getting a date is a pathway for other public health interventions.
- 23. THANK YOU! Clay Ewing (@claytical) c.ewing@miami.edu http://www.claytical.com Interactive Media Program http://interactive.miami.edu NERDLab http://nerdlab.miami
Notas do Editor
- CONTACTS: Clay Ewing c.ewing@miami.edu Dr. Soyoon Kim soyoon.kim@miami.edu This project was funded by the School of Communication at the University of Miami (http://com.Miami.edu) and the Center for Communication, Culture, and Change (http://com.miami.edu/center-for-ccc) Integrated Belief Model: https://www.sibson.com/media/1536/using-theory-to-design-effective-health-behavior-interventions.pdf Health Belief Model: http://foresttwpseniorcenter.com/5695732-the-health-belief-model-and-personal-health-behavior.pdf
- Hello! I’m Clay Ewing and I’m an associate professor at the University of Miami. I’m the Co-Director of NERDLab, where students and faculty work together on social impact projects that are primarily game based. Most of the student members of NERDLab come from our MFA program in Interactive Media. At the School of Communication, we also offer a PhD in Communication. Our PhD program has a strong health communication influence, so it was only a matter of time before we started working together on evidence based games for public health. Today I’m going to talk about a game I designed and developed with my colleague Dr. Soyoon Kim. Dr. Kim is a health communication researcher and together we worked to integrate theoretical frameworks into gameplay in order to change tanning intentions. But before I get to that, I want to step back in time to provide a bit of context for my relationship with measuring impact and how my experience evaluating games has evolved.
- In 2010, I was here in New York at Parsons finishing up my MFA in Design and Technology. My thesis project was Hustling Health Care, a board game that spoke to the debate about health insurance before the Affordable Care Act had been passed. My feeling at the time was fairly simple. If you presented players with an argument through a system, they would understand it better. If you used real world data, as I was doing, a player would come away with hard facts as well. And if the game was engaging enough, it could perhaps spark a deeper conversation. The game was playtested thoroughly. It was definitely engaging. I can remember Nick Fortugno coming by after this picture was taken and saying something to the effect of “If you can keep 4 high school students playing your game for 2 hours in the middle of Playtech, you have succeeded.” If you’re not familiar with Playtech, it’s an annual event held at Parsons where a bunch of kids are brought in to playtest a wide variety of games and experiences in development. It’s basically a big free arcade.
- Hustling Health Care pushed me towards social impact game design as a career. When my partner Lien Tran and I were asked to design a game for the World Bank and the Tanzanian government, it felt like we were moving towards something. The Tanzanian government had created the Tanzanian Social Action Fund with the objective of enabling poor households to increase incomes and opportunities while improving consumption. The game we designed set out to teach the TASAF staff about the program. It was well received and the staff suggested that the game could be modified to teach the community about the program as well. When it was piloted, the game had a debrief that reinforced ideas and concepts as well as asking about engagement and enjoyment. In certain contexts, this would be a sufficient evaluation, but the for study I’m focusing on today, the scope is much more specific.
- The thing that drives me to make social impact games is that they have the possibility of making a difference. When Paolo Pedercini commented in 2014 at G4C “If you can measure it, then it’s not the change I want to see… if your game or technology really works, it freaks me out”, I felt a bit of “Yeah, but…” Weaponizing games as a propaganda is scary, but just as film and radio continue to be used effectively, it’s inevitable that games will be used as well. Understanding the way in which they can persuade us is important otherwise we’re bound to be manipulated. Also, we use film and TV to deliver persuasive arguments for prosocial purposes. A good example would be East Los High on Hulu. Embedded in East Los are educational narratives that promote sexual and reproductive health. It makes sense to do something like this with games as well.
- At the same time we were developing the game for TASAF, we started working on a tabletop game funded by the Canadian Dermatology Association called Vanity. The game sought to prevent use of indoor tanning beds, particularly at an early age. We designed the game with a dermatologist as the content expert. There was no evaluation plan but as designers we felt that working with content experts would ensure the game was accurate and provide good information. When we started working with researchers at UM, Vanity was selected to evaluate from a health communication perspective. Our preliminary results were promising, but collecting data has been incredibly slow as the game requires a minimum of 3 players and more than an hour of play.
- Thinking about positive change and previous projects, I wanted to make something with a much greater reach. I had been working an app with ROC United on a Diners guide for ethical eating and one of the things that struck me was that most restaurant workers don’t receive paid sick days. Following my feeling with Hustling Health Care, I sought to create a game that advocated for restaurant workers by allowing the player to walk in their shoes for a month, working a restaurant making minimum wage and getting sick. But then…
- I was here at Games for Change and during her presentation, Colleen Macklin mentioned this article in Psychology Today. [READ QUOTE] The article is about a game called Spent. Is anyone familiar with Spent? I’m a big fan of Spent, I love it using it as an example in my design classes. So hearing that a well designed game had no effect on positive feelings toward the poor was interesting. But the fact that some participants who were sympathetic to the poor beforehand had a negative effect on attitude was troublesome. It freaked me out a bit. Since that Psychology Today article was published, the full study has been published in the journal Cyberpsychology. As an FYI, one of the key takeaways was that playing the game wasn’t effective at positive change, but watching someone else play it was.
- I think that we often have trouble with evaluating games because we debate what is considered impact and what it means to properly evaluate a game. Educational games are probably the most straight forward as it’s pretty easy for the designers and researchers understand the test results before and after. Increases in test scores means that the game is effective at teaching a subject. With games for public health and communication, you’re dealing with intentions and attitudes. Through work with our colleagues on the research side, we were finding that games with a narrative focus seemed to align well with theoretical frameworks in communication as where games focused more on play were a different beast.
- The games research group in our school came to the conclusion that a tanning intervention game was a good idea but as a tabletop requiring multiple players, it was requiring too much time to collect data. So now that you have a bit of context from my previous work, let’s dive into Dreamy. Dreamy is a single player dating simulator. It’s also digital. As the first game at the University of Miami’s School of Communication to be developed with a researcher from day one, we thought it was a great opportunity to match existing theoretical frameworks to the game mechanics. We chose two behavior prediction models: Fishbein’s integrated model of behavioral prediction and Rosenstock’s health belief model. The game mechanics target salient beliefs and attitudes to determine behavioral intention to engage in tanning behavior and discourage indoor tanning.
- The game has 9 conversation trees based on a 3x3 matrix of character combinations: outgoing, health conscious, and average paired with their attitude on tanning: anti, neutral and pro. In case you’re wondering, the way the each potential suitor looks is randomly generated. At the beginning of the game, a friendly guide asks about their gender preference to find matches. They can choose men, women or say that love has no boundaries. Based on this preference, the game generates the suitor avatars. Once the player starts a conversation with one of these characters, they won’t be able to talk to any other characters with tanning beliefs until they fail with the current conversation tree.
- You might also be wondering how all the characters end up talking about tanning. As the different characters have specific reasons for tanning or not tanning, we attempted to build conversations that would naturally end up mentioning tanning. There are dead end characters that never talk about tanning but the player can never get a date with them. In early playtests, we allowed the player to talk to multiple tanning suitors and while players didn’t think “This must be a tanning intervention” they did think the designers had some hangups about tanning. In this example, the player is talking to a character that is pro-tanning. You can see with the potential replies that the player is able to challenge the character’s beliefs with “I don’t think tanning beds are supposed to be used like that. Or at all.” Or they can agree with the character choosing “I’ve never thought of a tanning bed as being so corrective! That’s smart!”
- In the game, the player is invited to visit a tanning salon to improve their physical attractiveness. The player’s avatar can burn and even get moles. If they choose to go to the tanning salon, a dermatologist sends them a message offering a free check up. The dermatologist gives accurate information regarding tanning beds and skin cancer.
- To win the game, the player has to navigate successfully through a character’s conversation tree to get a date. If they date someone with a tanning history or engage in tanning behavior, their epilogue changes. There are a series of outcomes for both the player and their love interest that include having a brush with death due to skin cancer and in some cases, dying from skin cancer.
- So what did we find out? Our target demographic was young women, so in that aspect, the game can be said to have a positive impact because it reduced their intent to tan. They really didn’t enjoy the game though.
- Men and people who have indoor tanned enjoyed the game. However, they were more likely to tan after playing the game. That’s the exact opposite of what we wanted.
- We went really deep on data collection and intervention design. The results I’m sharing today are from one study but we will have quite a bit more as the research team analyzes all of the data. It’s also a problem figuring out what specifically caused the boomerang effect because there are many variables: how many characters they interacted with and which characters they interacted with, along with marketing offers, the tanning salon and the dermatologist. There are 117 health specific messages alone from just the suitors in the game.
- Transportation refers the feeling of being so absorbed in a story that the connection to the real world is lost for some time. One assumption that we’ve had for games is that increased transportation is a good thing. We’re not sure about this and we need more data, specifically limiting the amount of health messages the player receives. During the development process, Chun Zhou, one of our doctoral students, decided to use the game for her dissertation. She created a modified version to study interactivity. In the game, you’re able to construct your avatar, so she created a version that removed the avatar creation process as well as streamlined the conversations so that players choices were predetermined. Her initial findings have shown that story based interactivity is more effective than character based interactivity.
- After working on Dreamy, I’ve been thinking a lot about impact and how we talk about games for change. Dreamy was definitely impactful. By playing the game, it will change a person’s intent to tan. But not always in the right direction. The most important takeaway for me is thinking deeply about failure. We tend to celebrate and talk about games that either feel like they’re creating impact or are proven to have impact, but we shy away from telling people about the ones that didn’t come out the way we expected and might have unintended consequences and effects. We need to discuss our failures more.
- The exciting part is that we can iterate and figure out what works. We can also analyze the things that didn’t work as expected and try to figure out what happened so we don’t do it again.
- Our first step in figuring out the boomerang effect is to simplify the game and to do this we’re going to create multiple versions that focus on specific categories. For example, would our results change if we removed all pro-tanning characters? What if the game pushed every player to tan but then had a dermatologist intervene?
- For the designers in the room, you probably know what I’m talking about when I say that working with researchers can setup some hurdles. One of the bigger ones is finding common language. I’ve found that sometimes, envisioning a game through a design brief isn’t enough for a researcher to wrap their head around an idea. But prototyping a game just to explain an idea is a big ask of the developers. It’s much easier now to talk about using the skeleton of Dreamy where players talk to suitors with embedded health messages to think about other possible interventions.
- I’m a designer and I’m always happy to talk shop. Although if you are interested in the research side, I’m happy to connect you to my colleagues for further information regarding the nitty gritty of the studies. And just in case you’re interested in more information on our MFA program and our lab, here are the URLs. If there are any questions/comments and we have time, fire away. Thank you!