Earlier this year 2019, #NewMR and Wizu partnered to conduct an experiment with a chatbot. For our experiment, we conducted a chat survey looking at the Threats and Opportunities facing market research and insights.
We were able to collect 336 responses from people involved in market research and insights, along with quite a lot of material from online discussions about the project.
This webinar is one of two in the 'NewMR Chatbot Experiment – Threats & Opportunities Facing Market Research' series that provides feedback on the experiment.
The two webinars are:
- Sue York from NewMR and Martin Powton from Wizu look at the feedback about the chatbot experience (these slides)
- Ray Poynter from NewMR reports back on what the research said about the Threats and Opportunities facing market research.
The recordings from both webinars are available via the NewMR website Play Again page here: https://newmr.org/play-again/newmr-chatbot-experiment-report-back/
The NewMR written report will also be shared shortly.
6. Chatbot Look and Feel – 3 (flawed)
Question Answer
How optimistic Optimistic
Why feel that? I love what I do!
Main threat? DIY researchers with no training
Other threats? IT folks doing research
Main Opportunity? Don’t know
Other Opportunities? See prior answer
Any comment on AI, neuroscience or Predictive Analytics No
Other comments on this topic or survey You asked me a question that was not necessary
A fail, but not an epic fail
Happens in conventional surveys too
M
10. Use Cases
• Strongest
• Where conversation is predictable
• Customer Satisfaction, Employee Experience, Retrospective Experience Survey, UX
testing
• Cases where survey fatigue is high or where the risk of alienating/boring the
participant is higher
• For participants who reject conventional surveys – e.g. some younger people
preferring a chat on their mobile to a survey on their computer
• Most Challenging
• Exploring situations
• Diverse domains
• For example, now we have done this MR study we could program a better project
13. Wizu Comparison Study M
• 118% more words than traditional survey forms
• 91% more actionable insight
• 57% rated conversational experience as 10/10 – only 11% gave the
top score for traditional form based surveys
• All age groups and genders expressed a clear preference for
conversational surveys over form based surveys
• Did not bias core metrics
14. Learnings From the Project
• More qualitative research in advance
• Adopting a more iterative approach
• Technology was mentioned quite often, so the probes could have been tweaked to
ask people to specify which technologies
• The icon was considered not sufficiently business-like by some
• Perhaps use the NewMR logo or the researcher’s image?
• This was positioned as an experiment, that probably influenced the
reaction to it
• The open-ends from most people were very rich, even though we don’t
have a control cell to compare with
• How familiar are your participants with interacting with existing
conversational UI?
S