UCA: Data Gathering Techniques. Selection criteria
1. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
Introduction to User-Centred Analysis (UCA)
Data Gathering Techniques
Selection parameters
Cristina Cachero
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
2. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
The technique selection depends on several parameters:
1. Nature of the data that needs to be collected (task flow, features needed,
performance, subjective perceptions, etc.)
2. Number of people from which data must be collected (less than 25/more than 25?)
3. Do we need specific data or exploration data?
4. Do we need to gather data about any complex issue?
5. How quickly are data needed?
6. Which are our budget restrictions?
7. How much data do we need?
8. Do we need information about user errors?
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
2
User-Centred Data Gathering Techniques
3. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
Some companies provide cheatsheet to help in the
decision of which technique to use
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
3
User-Centred Data Gathering Techniques
E.g. We need to collect data about the features needed for the new application to
better support the customer service of our company.
This would incline us towards working with a direct method, where you can probe the user. Surveys could be
another option.
We have 10 people available to work with.
This would discard surveys, since there are too few subjects to make it cost-worthy.
This is the beginning of the study, so we are interested in exploratory data
Again, this situation inclines us towards more conversational-like methods, such as interviews or observation
followed by discussion.
There are some complex issues involved in the service (complex task flows with
many exceptions)
This strongly suggests the use of interviews and/or observation, where we can ask for further clarifications when
needed.
Particularly, data on user errors while using the actual system (which is preventing
customer service workers from giving correct solutions to the users) is needed
This would make it advisable to use observation, since users are not prone to reporting their own errors... many
times the are not even aware of them!
Source: HFI
4. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
4
User-Centred Data Gathering Techniques
Imagine that our software application to support the
customer service is being redesigned in an iterative
manner.The goal of the first increment consists in
fixing the most important complaints about the
current design.
Data gathering technique?
5. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
5
User-Centred Data Gathering Techniques
We want to develop an on-line tool to help people
decide when to go to the emergency room in a
hospital and when to wait for the doctor to fix an
appointment with them.
Data gathering technique?
6. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
6
User-Centred Data Gathering Techniques
There is a lot of people coming to your e-commerce
website, but few of them are buying and turning into
customers.
Data gathering technique?
7. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
7
User-Centred Data Gathering Techniques
We want to design a data entry device for ambulance
workers for them to register what happened in each
call and check the patient’s medical records.
Data gathering technique?
8. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
8
User-Centred Data Gathering Techniques
We have an internal operations application that is so
difficult to use that important company processes are
being ignored, critical data are not being captured, and
so on.
Data gathering technique?
9. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
9
User-Centred Data Gathering Techniques
Solutions (not unique!):
Customer service: Help desk reports (what information do they need to
gather and in which format?), then interviews and observation
Emergency room: Interviews for doctors and surveys to get the
correct names for diseases (using medical jargon may confuse
users)
E-commerce web site: Web analytics, then interviews or user
observation
Ambulance data-entry device: Interviews (complex process
involved, observation may not be possible)
Internal operations application: Interviews (complex issues
probably involved)
10. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
10
Importance of choosing the right technique:
The Wallmart case of study
Read the following case of study:
http://goodexperience.com/blog/2011/04/ignore-the-customer-e.php
11. Leonardo da Vinci Partnerships Project
GUI USABILITY AND ACCESSIBILITY:
EXCHANGING KNOWLEDGE AND EXPERIENCES
These slides are made available under the license Creative Commons
Attribution-NonCommercial-NoDerivs CC BY-NC-ND. More
information about license: http://creativecommons.org/licenses/by-nc-
nd/3.0/.
These slides were created under Leonardo daVinci Partnerships
Project 2012-1-PL1-LEO04-28181 GUI USABILITY AND
ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES
(http://usability-accessibility.org/).
This project has been funded with support from the European
Commission under the Lifelong Learning Programme
11
Attributions
Notas do Editor
Hello, my name is Cristina Cachero, and I am associate professor at the University of Alicante.
In this video we are going to dive into the parameters that influence the selection of one data gathering technique or another.
This selection of the most suitable technique depends on several parameters: which data need to be collected? How many people will be involved in the research? Do we need specific data (what is exactly happening, how, when and where) or exploration data, more qualitative in nature (for example, why questions). How complex is the issue we are trying to understand? Which are our budget restrictions? Do we need to collect information about user errors or misconceptions?
Answering these and other questions can help us create a decision tree to select the most appropriate technique.
Some companies, such as Human Factors International (www.hfi.com), in an attempt to simplify this decision process for novice usability analysts have created a cheatsheet that, for a number of methods, indicates their suitability depending to the answer to these questions.
For example, imagine that we need to collect data about the features needed for the new application to better support the customer service of our company.This would incline us towards working with a direct method, where you can probe the user. Surveys could be another option.
If we now add that we have 10 people available to work with, this would discard surveys, since there are too few subjects to make it cost-worthy.
Also imagine that this is the beginning of the study, so we are interested in exploratory data. Again, this situation inclines us towards more conversational-like methods, such as interviews or observation followed by discussion.
We can assume that there are some complex issues involved in the service (complex task flows with many exceptions). This strongly suggests the use of interviews and/or observation, where we can ask for further clarifications when needed.
If, furthermore, data on user errors while using the actual system (which is preventing customer service workers from giving correct solutions to the users) is needed, this would make it advisable to use observation, since users are not prone to reporting their own errors... many times the are not even aware of them!
Now let's work with some examples that help us further get acquainted with the different data gathering techniques mentioned in this video. Imagine that our software application to support the customer service is being redesigned in an iterative manner. The goal of the first increment consists in fixing the most important complaints about the current design. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
Now imagine we want to develop an on-line tool to help people decide when to go to the emergency room in a hospital and when to wait for the doctor to fix an appointment with them. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
In our third setting, imagine that there is a lot of people coming to your e-commerce website, but few of them are buying and turning into customers. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
In our fourth example imagine that we want to design a data entry device for ambulance workers for them to register what happened in each call and check the patient’s medical records. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
The last situation involves an internal operations application that is so difficult to use that important company processes are being ignored, critical data are not being captured, and so on. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
Here are my selection of techniques for the situations exposed. Please, take into account that there are many nuances in the context that may cause the selection of techniques to change. So there is not absolute right or wrong answer. And remember, the best way to go is to try to combine different data gathering techniques so that the reliability of your data increases! .
In the customer service app, my first option would be to go through the existing reports they issue, in order to get to grips with the exact information they need to gather and how they summarize it. Then, I would try to interview some customer service workers to check what is working and where do they find difficulties making the system adapt to their needs, and finally I would try to observe at least a couple of workers as the work. Customer Service (and, in general, any job post that requires direct contact with customers) is subject to lots of stress, and therefore understanding the work environment under which those workers operate may change the way in which we tackle the redesign of the user interface.
Following the same kind of reasoning, for the app to decide whether to go or not to go to the emergency room I would interview doctors (few) and run surveys on patients (many) in order to get the exact language for the application right.
Regarding the e-commerce web site, the first (and most economic-effective) thing to do would be to run any kind of web analytics, to check where customers are abandoning the purchase process. Then, in order to answer why they are leaving, I would go for interviews or user observation.
For the design of an ambulance data-entry device, I would use interviews, since there is a complex process involved, and getting into the ambulance to observe the whole process may not be possible.
Last but not least, for the internal operations application I would run a round of interviews, since the complexity of the issue is high.
The importance of selecting the right technique cannot be overstated. As a real example of how not understanding the possible biases introduced by the different techniques can lead to billionaire losses, please read the Wallmart case of study. Remember to take some notes, since you will be asked about it in the quizz that accompanies this part of the course!
So, this is all for now. Hope you have enjoyed this video. See you in next section!