User experience interviews - How qualitative user feedback improves your business User experience interviews are a great way to extract qualitative feedback from users for user experience and usability understanding. Information can be extracted from users about a certain application, product, service or even an idea. User experience interviews may therefore uncover possible issues and can give the interviewer valuable user arguments, problems and questions.

1. User Experience
Interviews
How qualitative customer input
improves your business
Peter Gevaerts, 24 May 2018
User Experience Interviews - How qualitative customer input improves your business
- Observing and interviewing customers is a great way to gather qualitative input for your project. You don’t want to end up developing something that will not be used
by anyone.
- Customer sessions bring the voice, the behaviour, the needs and the fears of users to the work floor.
- To retrieve valuable insights from users, there are some common practices to be respected. Otherwise you might only hear yourself talking about that great
business idea of your boss. Today I will tell you about this common practices in a few clear steps.

2. GDPR communication & settings
UX of different levels,
some good, some rather poor
- Tomorrow is the big day for GDPR. To better protect the privacy rights of citizens.
- You certainly have received many requests to re-new your consent. Some were OK, some were not at all OK. How does it come, you think?
- It was not easy, because the people who make up this regulations are not UX people, neither are they copywriters.
- Business owners can be afraid of customers opting out, refusing specific cookies, holding back on providing personal data …. when they are given too much
transparency and liberty …. Some companies are only implementing the minimal interpretation to be compliant with the new regulations
- The once that were good provided clear language, understandable options and a simple structure, they certainly did interviews with customers.

3. On-train communication system
- Communication required
between train driver,
supervisors and train station
- Not spoken to end users?
- No field research?
- Project canceled
- €11 million taxes spent
- An on-train communication system that was in development.
- The system was clearly not adapted to the users of it.
- The project was canceled: 11 million euro of taxes was wasted.

4. An appointment to buy
high-end electronics
Business idea
- In-store appointment for selection
of high-end goods
UX interviews
- No interest from customers
- Project killed in ideation
- Without interviewing it would have
been build
- Business idea: Let people book an appointment with a salesperson at an electronics store for the right selection of high-end goods.
- The business owner was convinced this was a good idea, but there was no interest, no need at all from customers.
- Without interviewing customers in the ideation phase, it would have been developed.
- The investment was saved, from building a service that nobody was waiting for.

5. Credit to pre-finance corporation tax
The request flow tested great,
but the product was not understood…
- Easy flow to make a simulation and to get an online approval for a credit
- The screens of the flow to request the credit tested very well.
- But people had no clue about the product: to pay for a credit in order to pre-finance and receive a discount on my corporate taxes? Why would I do that? Is that
worth the extra cost of the credit?
- Unfortunately, explaining the product was out of project scope.

6. - To avoid this situations, we should keep Design Thinking in mind.
- It tells us to include the end-users in the pre-analysis and different validating phases. In different iterations, the more the better.
- It is not only about the interfaces, it’s about the full context throughout the whole customer journey.
- The end user has to be seen as one of the stakeholders, thinking human centred design …
- So, do not only talk to the business owner, because customer value = business value.

7. Nice landing, wrong airport
- Because you don’t want to make a great project and develop services that will not be used, when it is not adapted to the customer needs.

8. UX Interviews
1. Prepare
2. Interview
3. Report
4. Follow up
- Today I will explain how to conduct UX interviews in 4 steps.

9. Qualitative feedback about a concept, feature, application,
about the competition …?
What is the research need?
- What does the team want to learn from the research?
- A check on some new technology?
- The behaviour in using a product or service? The paint points?
- To get to know the target audience? Their needs, fears, limitations….
- Check on the usability?
- Understand your users, who are they, what are the trying to accomplish?
- Qualitative feedback, evaluate existing system, what is currently working/not working?
- It’s about the psychology of users, about their conscious and unconscious behaviour, influenced by the context, background and earlier experiences, about the
difference in culture …
- Divert to other research methodology if needed.

10. Are there any existing insights?
Consider other research material
Consider existing research material from,
- A/B testing
- Web analytics
- Persona’s
- Card sorting
- Focus groups
- Market surveys …

11. Interviewing in context is a very valuable environment
Where to interview?
- In context will tell you about your user in his/her environment. Taking your research to where people are instead of bringing users to you.
- Or in a regular meeting room. Not too intimidating, practical for participants, parking, public transport …
- Motivate for observation
- Observation: In-person? (1 person max)
- On tv-screen next doors or at separate location?

12. Define the profile of your interview audience
Ask professionals for the recruitment
- Customer / non-customer
- Specific product experience?
- Specific device or OS requirements?
- A diverse mix of participants within the target group: age, gender, profession, experience level ….
- This is qualitative testing: with 5 to 8 people, you cover about 80% of the major topics and issues.
- Location, timeslots & incentive
- Best use a field marketing agency for recruitment. It’s a specific job you don’t want to spend time on.
- Start about a week ahead.

13. The better the level of detail and maturity,
the more precise the insights
Test material detail & maturity
- Format: Live website? Clickable prototype, QA, paper prototypes…
- Tangible material is easier for participants to test.
- Agree on the expected level of detail, copywriting, language, consistency ….
- High level wireframes make it difficult, they need more effort for ‘interpretation’.
- Real content and copy is preferred.
- The better the maturity, the more precise the insights.

14. Write down tasks
- Collect information to gain insight to preferences, usage, mental models.
‘You’re looking for a credit card. Compare the different offers.’
‘Order a specific product and register if required.’
No leading questions, avoid a Yes or a No
‘How was it to… ?’ instead of ‘Was it easy?’
‘Why was it so and so … ? Can you explain me?’
Test scenario’s
- You want to get qualitative feedback about an idea, a feature, an application, certain technology, about a competitor …
- Therefore you want to delve deep into the thought process of the participant.
- You want to get to know your audience and their context.
- No leading questions that guide people into an answer.
- Ask open questions, not to be answered with Yes or No.

15. Do the test of the test for real !
Not just quick and dirty
Dry run
- Test all scenario’s for real with a colleague
- You will find pitfalls in your scenario’s
- You’ll find weak spots in the test material Otherwise, your first interview might be wasted!
- Now you still can get the material corrected.
- People are sensible to details and will point it out. Each typo or illogical content will be a hick-up during the test
- You’ll have to come up with an explanation, an excuse … and you will have to bring the user’s focus back to the test subject … all for something you are not looking
for!

16. UX Interviews
1. Prepare
2. Interview
3. Report
4. Follow up

17. Coffee, chocolate … parking voucher … guest list …
Practical preparation
- Make it a fine environment, as you would have visitors at your home.
- Foresee a reception for your guests, not to be disturbed during a test.
- Non-disclosure is important, that’s evident.
- Permission for recordings, keep the privacy regulations in mind.
- Incentives, to give at the end of the interview.
- Test the technical set-up.

18. Create a positive mindset in
order to have your participant
open up
Ease the participant
- First some limited small talk to relax.
- Welcome your participant and introduce yourself.
- Explain how things will happen.
- Tell that it is not your work that will be judged, so they express their most natural reactions.
- Tell them they could hold back to a certain task at any time. Sometimes things might become too personal (money, divorce …).
- (The recruitment agency should have explained this beforehand, so this should not happen of course)
- So create a positive and open atmosphere that will ease your test participant to open up and talk to you freely.
- Run through their profile + add some extra questions to better know their situation and as introduction to the topic.
- Think aloud: ask to talk while they are thinking. That is not as easy for everybody.

19. Recording interviews is
like magic
Note taking?
- Either have a separate note taker or record the sessions.
- Recording of device and/or user face?
- Writing notes holds up a natural conversation
- Typing on your laptop creates a wall in between you both
- A note taker or recording lets you focus on the conversation,
- Focus on every word of the conversation. Because every word you say is important and influences the answers.
- There is easy tools for recording: audio, screen capture, face capture (Quick Time, Lookback, Go to Meeting, Screenhunter …)

20. Let there be silence
Interview style
- Give time for thinking, not everybody is as fast, most people really want to do their best for you.
- The test scenario is not a telesales transcript that has to be rattled off.
- It is the backbone of the interview, it’s not to be taken literal: every conversation goes differently.
- Experience will tell you how much you can let the conversation go, before you return to the scenario.
- Recognition and confirmation can lead behaviour or interrupt the thinking flow of the participant. You better say: ‘Interesting, tell me more.’
- The interviewee asks you to explain how something works. Instead of answering, reply with ‘How do you think it works?’
- Do not talk about yourself, do not join the conversation, stay the researcher at all times.
- Do not reinforce your own opinions.
- No social talk. You’re there to gain information, not to establish friendships.

21. It’s all about
observation
Interview style
- It’s about observing people, to understand their behaviour. Opinions are of 2nd importance, because people act different then they say they do.
- ‘Disappear’, you become wallpaper, you are not longer there once a task is been given.
- Notice the body language, it will help you to better understand (crossed arms, open or closed …).

22. UX Interviews
1. Prepare
2. Interview
3. Report
4. Follow up

23. Elements of a report
Report format according to the project needs
- Formats: bullet point list, (management) presentation, full text, best-off video clips, complete interview transcript …
- The quicker the iterations go, the more efficient the reporting has to be.
- Instead of full transcript, invite your customer to attend the sessions.
- Mandatory items: research question, executive summary, test set-up / context of the test, screenshots / link to prototype, anonymous profile data of participants, test
scenario.
- A report will start living it’s on life, you have no control on who’s desk it will land. So it should stand on it’s own. Understood by people who were not there.
- Always delete personal data and recordings of participants when done with the report.

24. Translate detailed test results
into an aggregated insight
- We look for patterns in behaviour: you have turn the detailed test results into aggregated insights.
- Test participants do not tell us literally the solutions, they make alarm bells go off.
- Neither do we design for one person, so be careful for individual behaviour.
- Also be very careful with going in the quantitative way: 5 to 8 people are not a relevant statistical basis.

25. Keep your readers in mind
Present insights in an understandable way
- Keep it efficient: distillate from notes and/or recordings what is relevant for the project scope.
- Prioritisation of insights
- Quotes are speaking
- Don’t write full books.

26. UX Interviews
1. Prepare
2. Interview
3. Report
4. Follow up

27. Bring a testimonial of your interviews,
make it a workshop with the whole project team
Don’t simply deliver the report, bring your message across
- One of the hardest parts is a smart interpretation of insights, to tweak your service in a better direction. Insights cannot be copy/pasted directly into solutions
- There is not 1 solution: interviews provide us with insights towards several possible solutions, to be judged by the project team.
- Research makes the pain points visible, it makes the good parts stand out.
- Don’t just send the report telling that ‘you are available for any questions’. That very often does not happen spontaneously.
- The researcher brings the message across in person. Go through the insights together with the project team. Present your insights, bring testimonials, have a
workshop. Discuss possible solutions together.

28. Go back to the project team,
they certainly have new questions …
Follow up
- Go back to the project team after a while.
- What have they done with the insights?
- How did they deal with the pain points?
- It happens that little is done with the insights, the team is too busy and deadlines are approaching, simply delivering is more important than delivering the right
thing ….
- This are all excuses to cover up the fact that it is too hard for them to really do something with it.
- Keep them motivated to get back to you. Make sure they see you as an added value to their project. Propose more research if needed.

29. Team-players
Conclusion: UX Interviews & BA belong together
- Projects that have solid analysis and up-front research are much more likely to reach their completion successfully meeting both business needs and end-user
needs. They also see a reduction in the overall time required.
- So don’t play the game of ‘us and them, because the UX researcher and the business analyst are brothers in arms for successful projects, each with their focus, but
with a common understanding and goal.
- So it’s time to get out of the office and get to know your users!!

30. THANKS
FOR LISTENING
ACA MOBILE
Mobile Enterprise
Solutions
ACA CREATIVE TEAM
Design & Marketing
AMPLIFLY.
Atlassian Solutions
IT MATCH
Outsourcing, Consultancy
& Selection
COIN
Content & Information
Solutions
COLLECTIV
Java Software &
Consultancy
OPSKLAAR
Cloud Solutions
Questions?