O SlideShare utiliza cookies para otimizar a funcionalidade e o desempenho do site, assim como para apresentar publicidade mais relevante aos nossos usuários. Se você continuar a navegar o site, você aceita o uso de cookies. Leia nosso Contrato do Usuário e nossa Política de Privacidade.
O SlideShare utiliza cookies para otimizar a funcionalidade e o desempenho do site, assim como para apresentar publicidade mais relevante aos nossos usuários. Se você continuar a utilizar o site, você aceita o uso de cookies. Leia nossa Política de Privacidade e nosso Contrato do Usuário para obter mais detalhes.
An update to my Introduction to Games User Research lecture (http://www.slideshare.net/Gortag/an-introduction-to-games-user-research-methods). Due to a changing course design this version focuses a bit more on questionnaire design and interviews. A few other changes have been made and the aesthetics have also been changed.
GUR Examining the player Fun & user experiences
Fun What is fun? Easy to use Challenging Emotional impact Engaging Compelling Relaxing It is subjective!
Sometimes fun High Activation Often fun Unpleasant Pleasant Low ActivationAlmost never fun
General tips Get (enough) representative users The game is being tested, NOT the user Work out what you want to know
General tips Test as early as possible, fix problems, & then test again (RITE) Listen to problems, not necessarily solutions Not (primarily) for balance & bugs, but for fun! Impact evaluation is different from general GUR
Focus groups 6-10 people Lead by a facilitator Specific questions Try the game/discuss potential ideas Talk about it
Focus Groups Pros More people can = more feedback Gets everyone together in one place Follow up questions Good for discussing concepts Cons You need a good facilitator Strong voices may take over Too many “helpful” suggestions What people say is not often what they do Not for evaluation of impact
Heuristic Evaluation Expert evaluation (very loosely like a game review) Are clear goals provided? Are players rewards meaningful? Does the player feel in control? Is the game and the outcome Is the game balanced? fair? Is the first playthrough and first Is the game replayable? impression good? Is the AI Is there a good story? visible, consistent, yet Does the game continue to somewhat unpredictable? progress well? Is the game too frustrating? Is the game consistent and Is the learning curve too steep responsive? or too long? Is it clear why a player failed? Emotional impact? Are their variable difficulty Not too much boring levels? repetition? Can players recognise important elements on screen? (Christina et al 2009)
Heuristic Evaluation Pros Smaller numbers Experts are experts Cons You need experts Which heuristics to pick? Experts are experts
Questionnaires &surveys During gameplay (at or after set moments) After gameplay Ask for what interests you Allow for some open ended answers
I can’t show the whole scale here. If you want the GEQ go to http://www.gamexplab.nl/Game Experience Questionnaire (GEQ)http://www.gamexplab.nl/
How to design aquestionnaire 1. Work out what you want to know 2. Design the questions/statements 3. Create the questionnaire 4. Test the questionnaire
1. Work out what you want to know Brainstorm, make lists, look at your design documents Then cut it down Only what you need to know Aim for no more than 15 minutes
2. Design the questions/statements Questions vs StatementsQuestions are good for gaininginformation (age, ratings of fun,etc)How much fun did you have in the last quest? 1–2–3–4–5-6-7 None A lot
2. Design the questions/statements Questions vs StatementsStatements are good forassessing attitudes i.e. agreement with an ideaThe last quest was a lot of fun 1–2–3–4–5-6-7 Strongly Strongly agree disagree
2. Design the questions/statements Use clear, everyday language I really felt like I identified with the motivations of the main protagonist in the game The main character in the game was just like me
Can players identifythese enemies byname?Consider visual aids
Without images would your playersknow a M4A1 from a G36C?
2. Design the questions/statements Closed or Open?Indicate your agreement with the followingstatementThis is one of the best research methodslectures I have ever attended: 1 2 3 4 5 Strongly – Agree – Neutral – Disagree – Strongly Agree Disagree
2. Design the questions/statements Closed or Open?What was your favourite weapon, and why?________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Yes/No, Male/Female, True/False Direct & precise, not very data rich2. Design the questions/statements Scale: Dichotomous
Place a mark along a continuum Data rich, a bit vague2. Design the questions/statements Scale: Continuous
2. Design the questions/statements Scale Be consistent Problem with lazy responding? Use a trick question/statement e.g. “Agree with this statement” - Use even spacing 1 2 3 4 5 1 2 3 4 5
2. Design the questions/statements Both the question/statement and the scale should be clear, and exhaustiveWhat education do you have? Primary School Secondary School Bachelors Masters
2. Design the questions/statementsWhat is your highest completed level of education? Primary School Secondary School Trade School Bachelors Masters PhD Other:______________
2. Design the questions/statements Free vs Forced choice What game types do you enjoy (select as many as you like) vs What is your favourite game type (select one)
2. Design the questions/statements Clear, and exhaustive Only a single concept/question Not leading “This game was fun, how fun was it?” Not loaded “People with bad co- ordination typically do badly in our game, how did you do?”
3. Create the questionnaire Paper vs digital Question order Number the questions Easy questions at the start Don’t lead/cue/give away later answers
Questionnaires & Surveys Pros Consistent Quantifiable Fast Cons Lack follow up Not objective Need a large(ish) sample
Interviews 1. Setup 2. During an interview 3. After the interview
Conducting an Interview 1. Setup Chose a good setting Comfortable with few distractions Only one or two people Explain why you are interviewing them Again, about the game, not them Tell them how long it will take Get permission to record the interview
Conducting an Interview1. Setup Have some set questions Ask follow up questions Don’t be confrontational
Conducting an Interview 1. Setup The same rules as for in a questionnaires Not leading, neutral, not loaded, clear, only one meaning Avoid questions that can be answered with yes or no Start with easy questions
Conducting an Interview 2. During the interview Only one question at a time Encourage responding Head nodding, “ah huh” But don’t let them go off on a tangent for too long
Conducting an Interview 2. During the interview Be as neutral as possible “Act as if you have heard it all before” Careful with note taking People are suspicious of interviewers End with an opportunity for interviewees to add anything they would like
Conducting an Interview 3. After the interview If you have time make a transcript Look for common threads in what people say Group into themes May point issues you may have never expected
Interviews Pros Rich data Can follow up Good for scope Cons Less quantifiable Time consuming Not objective
Observation studies Watch/Record Either with a facilitator or without Facilitator must be as hands off as possible Watch faces/body for emotion Only write down what you actually see!
>Test scripts Like a recipe for a delicious cake, these lay out how the test will go Order of events What the tester should do and say, and when Clear and precise Could anyone follow the instructions and carry out a test? Ensures consistency!
Contextual Inquiry (fieldstudies) Similar to formal observational studies In the wild Observing real use in the real environment Better once a game is released Or similar games Useful for evaluation
Think out loud Observe gameplay, and note down what they say & when they say it Don’t prompt them, and don’t correct them
Observation studies Pros Objective data i.e. You see what players actually DO, not what they say they would do Facilitator can help if really needed Cons Time consuming to analyse video Training required to get the best out of observation (especially for emotion) Avoid Observer Bias
Gameplay Metrics Observation via game data Number of incidents Where, when, and how they occurred and with who or what?
Gameplay Metrics Pros Objective data Good for evaluation See trends Cons Time consuming No subjective feedback/context Needs larger sample sizes Data overload
Biometrics/psychophysiology Measuring body signals: From the Brain (EEG), the Heart (EKG), the muscles (EMG), the eyes (eyetracking), the skin (EDA), etc The body gives clues into cognition, and emotion
Biometrics/psychophysiology Pros Gives objective quantifiable data Allows for continuous data recording Cons Invasive Costs a lot of time & money to use & analyse Problems with specificity, artefacts, inference and validity
Summary Many options In most cases I recommend: Observation Objective data & insights After play (between level) questionnaires Specific questions & subjective feedback WARNING: People don’t often do what they say Metrics are great for evaluation, tweaking, issues, & insights
Summary Don’t wait until the game is almost finished for GUR It is easier to change & plan things early in the process Listen to what people say is wrong/right, don’t worry too much about what they suggest to do to fix it You are the game designer
Credits All the copyright holders of the images I have used A rough primer to user research (parts 1 & 2) http://www.gamasutra.com/view/fe ature/169069/ http://www.gamasutra.com/view/fe ature/170332/ Christina et al 2009 http://mi-lab.org/wp- content/blogs.dir/1/files/publicatio ns/uxInGames_Koeffel_et_al.pdf Special thanks to the GUR-SIG for their feedback and inspiration