O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Carregando…

Flash Player 9 (ou mais recente) é necessário para visualizar apresentações.
Nós detectamos que ele não está instalado no seu computador. Para instalá-lo, vá para here.

MG673 - Session 1

209 visualizações

Publicada em

Evaluating the User Experience

Publicada em: Educação
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

MG673 - Session 1

  1. 1. Evaluating User Experience MGA 673 – UX evaluation Sónia Sousa Tallinn University School of Digital Technologies
  2. 2. Why we need to evaluate?
  3. 3. Evaluation • Main idea • Get informed about the value of your product: • idea, conception, users’ needs, understandability, learnability, trust... • Getting a feedback about your design issues: • Addressing end users and collecting data about their perception and usage patterns. • To better understand users experiences, • Letting them to reflect on their needs and experiences • meanwhile trying to make sense about the context, stimuli and time factors Why we need to evaluate?
  4. 4. Purpose Evalu- ation Artefact Benefici- aries Interaction design
  5. 5. When?
  6. 6. Evaluation (When?) Development phase – Scenarios, sketches; i.e. concepts – Early prototypes – Functional prototypes – Products on market
  7. 7. Where?
  8. 8. Evaluation (Where?) Study type: – Field studies – Lab studies – Online studies – Questionnaires / Scales
  9. 9. Who to use?
  10. 10. Evaluation (Who to use?) Participant – UX experts – One user at a time – Groups of users – Pairs of users
  11. 11. What to evaluate?
  12. 12. Evaluation (What to evaluate?) Study period: – Before usage – Snapshots during interaction – An experience (of a task or activity) – Long-term UX
  13. 13. What Objects evaluate?
  14. 14. Evaluation (What to evaluate?) Interactive objects – Interface on a screen – Platform – Browser – Mobile – Spatial interactive objects Information exchange • Visual • Audible • Tangible
  15. 15. Evaluation • Let’s then take a look into concepts such as • Evaluation Concepts: usefulness, usability and user experience • Evaluation Methods and Tools – Objects for Evaluation – Methods & categories & metrics – Data & Scales 2016 Sónia Sousa 15
  16. 16. Evaluation Concepts: usefulness, usability and user experience
  17. 17. a good user experience depends on both pragmatic and hedonic qualities
  18. 18. Usability Interaction design User experience Useful concepts Pragmatic qualities Hedonic qualities
  19. 19. usefulness and usability are pragmatic
  20. 20. pleasure is hedonic
  21. 21. pleasure is hedonic
  22. 22. Usability Interaction design Useful concepts Pragmatic qualities the extent to which a product or service can be used by specific users to achieve predefined goals in a specified context of use How? Assessing products or service effectiveness, efficiency and satisfaction. Consider also learnability and memorability. Usefulness the degree to which a product enables a user to achieve his or her goals, How? assessing user’s willingness to use the product or service.
  23. 23. Interaction design User experience Useful concepts Hedonic qualities A person’s perception and responses of use or anticipated use of a product or service. How? Assessing additionally experiences emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use. User perception the degree to which a user perceives the product or service as providing a good experience, How? assessing user’s intentions to use the product or service.
  24. 24. Usability and user experience 2016 Sónia Sousa 24
  25. 25. How to…choose a specific method?
  26. 26. evaluating user experiences is tricky but possible
  27. 27. and we will now address possible approaches
  28. 28. Methods and Tools User Experience and Research
  29. 29. Methods & categories • Categories help to pick suitable method according to study conditions. 2016 Sónia Sousa 29 Methods • Formative • Summative • Design study • Experimental study • Comparative study Categories / study Type • Field studies • Lab studies • Online studies • Questionnaires / Scales
  30. 30. Methods & categories Categories / Time factor • Before usage • Snapshots during interaction • An experience episode (of a task or activity) • Long-term UX Method / Measures • Self reported • Observed • Directly measured (objective) • Latent value (calculated, concluded) – The "Big Five personality traits" have been inferred using factor analysis. • Expert analysis (opinion)
  31. 31. Data & scales Metrics / Data • Time to complete task • Count of errors and/or effort • Written report • Schemas/diagrams • Audio recordings • Video recordings • Images/photos • … Metrics / scales • Indicators – Effectiveness – Efficiency – Satisfaction • Likert scale • Comparison values • Visual values • Emotional values
  32. 32. Other conditions Other conditions • Conditions of use • Surrounding environment • Human factor – Prejudice – Previous experience • User profile • Demographic background • Emotional background • Experience level Outcome of UX study • What and How certain behaviours occurs • UX value • Explaining the issues – Why it occurs? – How to solve them? 2016 Sónia Sousa 32
  33. 33. Most known Methods… • Usability Methods • System Usability scale [SUS] • Methods that were transposed from Usability • Cognitive Walkthrough • Think aloud • Diaries 2014 Sónia Sousa & Mati Mottus 33
  34. 34. Most known Methods… • Others are adaptation of – Extended usability testing – TRUE Tracking Realtime User Experience – UX Expert evaluation – Property checklists • Others are new – AttrakDiff - – Emocards, Emotion Cards, Emofaces – Reaction Cards 2016 Sónia Sousa 34
  35. 35. SUS is a quick and dirty scale popular for end-of-test subjective assessments
  36. 36. System Usability Scale
  37. 37. for positively-worded items [1, 3, 5, 7, 9] the score contribution is the scale position minus 1 for negatively-worded items [2, 4, 6, 8, 10] it is 5 minus the scale position to get the overall SUS score, multiply the sum of the scores by 2.5
  38. 38. SUS scores range from 0 (worst) to 100 (better)
  39. 39. AttrackDiff
  40. 40. AttrackDiff assesses the user's feelings about a system with a questionnaire addressing both hedonic and pragmatic dimensions of UX
  41. 41. AttrackDiff explores… pragmatic qualities identity stimulation attractiveness
  42. 42. technical...human complicated...simple impractical...practical cumbersome...straightforward unpredictable...predictable confusing...clearly structured unruly...manageable
  43. 43. http://attrakdiff.de/
  44. 44. Inspection methods • There are also several but we will address: – Cognitive walk-through – Heuristic evaluation
  45. 45. Testing methods • There are several but we will address: – Co-discovery – Wizard of Oz
  46. 46. Benefits • This technique can be used to test device concepts and techniques and suggested functionality before it is implemented 48Image source: http://stanford.edu
  47. 47. Testing methods • This kind of evaluation should be repeated until no significant added value comes from bring in an additional subject – The usual number is 5, but other claim otherwise
  48. 48. Pre and post-tests • When applying testing methods, pre-test and post-test questionnaires are also used to gather feedback on the product being tested – A common questionnaire is the 25 years old System Usability Scale
  49. 49. Evaluation protocol • But these methods are useless without an adequate evaluation protocol • While designing your evaluation protocol, you should take into account… – A mixed selection of inspection and testing methods; – The user stories supported by your prototype; and – The affordances of your paper prototype.
  50. 50. What we discussed • Why we need to evaluate? • Main idea, purpose of evaluation • When and, Where we need to evaluate. • What to focus during the evaluation process? • Who shall we use? • The object of evaluation • Evaluation Concepts • Usefulness, Usability and User experience • Methods and Tools • How to choose a specific method of evaluation?
  51. 51. Future aims • Focus on Methods and tools • Design a evaluation procedure • Implementation protocols • How to Apply the most known methods • During the different development phases • Concept • Low fidelity • High fidelity
  52. 52. In sum… • As mentioned before • In general, product and service evaluation can be formative or summative • Low-Fi prototype evaluation is mostly formative • Not so rigorous procedure of implementation • Hi-Fi evaluation • Include both formative or summative • The method for data collection should be mixed • The evaluation procedure should be carefully planed • The quality of results depend on it

×