Conversational User Interfaces, Past and Future

Conversational User
Interfaces: Past and Future
Crispin Reedy @crispinTX

2017 goals
1. Provide more UX resources
to UX professionals.
2. Improve brand consistency.
3. Improve “International”
UXPA.
User Experience Professionals
Association (UXPA)
Monthly webinars
User Experience Magazine
The Journal of Usability Studies
Visit www.uxpa.org for more info
about the conference, webinars, and
other UX resources!
Annual Conference in
Toronto, Canada
June 5 – 8, 2017
uxpa2017.org

4© 2017 Versay Solutions LLC
• Voice User Interface Designer
• 15+ years in the field
• Former coder; got interested in UX
• President of the Association for Voice
Interaction Design
• Consultant for Versay Solutions
@crispinTX
crispinreedy.com

CUI: Past and Future
• How can chatbots learn from existing VUI
design? What makes these new interfaces
different, and how are they similar? Where
do the Alexas and Siris come into the mix?
We’ll discuss text-based vs. voice-based
conversational user interfaces, and the
landscape of Conversational User
Interfaces, now and into the future.

Disclaimers
• What, exactly, is a Conversational User
Interface?
– For this discussion we are going broad and
wide.
• Conversation:
– Interactive communication between two or
more people
• Spoken vs. written (chatbots)
• Turntaking vs. one shot

Flash-forward to 2017….

“I’m really excited about chat and natural
language because there’s no user interface
you have to deal with.”
- Chatbot writer on support board

“Just this week, Facebook said it was “refocusing” its use of AI
after its bots hit a failure rate of 70 percent,
meaning bots could only get to 30 percent of requests without
some sort of human intervention.”
“I would call it overpromising,” said CP+B executive creative tech director Joe Corr.
“Brands that created bots with a structured request or utility like Domino’s or in retail
were easy. But bots that tried to break out of the utility and be
chatbots became the problem.”

Chatbots
Voice
Systems
Taxonomy of CUIs
Input Modality

Chatbots
Voice
Systems
Taxonomy of CUIs
Input Modality
Domain Domain
•Personal Assistants
•Agents
•General Search
•Order Products
•Troubleshooting
•Manage Accounts
•Alexa
•Google Home
•Wildfire
•IVR
•Billing information
•Control-based

Voice
Systems
Taxonomy of CUIs
Domain
•Alexa
•Google Home
•Wildfire
•IVR
•Troubleshooting
•Manage Accounts
Technology
•What “thing” are you
talking to?
•What else are you
connected to?
(Network, data)
•What kind of speech
technology are you
using?
•What other input and
output modalities are
you coordinating
with?

What Are You Talking To?

Voice Conversation vs. Chat
Conversation
• Differences
– Written convention vs. spoken convention
– Speech recognition vs. textual understanding
• ASR vs. NLU
• DNN, Machine Learning
– Chatbot implies keyboard; voice interface
could be to almost anything
• This becomes really important:
– Discoverability
– Onboarding
– Coordinating voice with other modalities

Voice Conversation vs. Chat
Conversation
• Commonalities
– Logical structure of a conversation (flow)
– Error recovery
• If caller / user goes off topic
– Prompts elicit expected responses
• Sometimes!
– Conversational maxims
• English is still English! Well, sort of.

UX Still Applies!
• Typical Tools:
– Tasks
– Personas
– “Happy Path” High Level Conversations
(Script)
– Card sorts?
– Read Aloud
– Usability Testing
• Wizard of Oz
– A/B Testing

Voice Conversational Model: Beginning
NL Level 1:
Open Ended
Question
• Hi, how may I help you
today?
• I’m calling about a
problem with my phone.
• Sure, what kind of
problem are you
having?
• I don’t seem to have
any internet.
• I can help you with that.
Are you calling about
the Apple iPhone 6 that
you have on your plan?
• Yes
NL Level 2:
Identify Problem
Directed Dialog:
ID Phone
Implicit: ID Caller

Chat Conversational Model: Beginning
NL Level 1:
Open Ended
Question
• Hi, how may I help
you today?
• I have a prob w/
phone
• Sure, what kind of
problem?
• No internet connectin
• OK. Is that about the
Apple iPhone 6 you
have on your plan?
• Yep
NL Level 2:
Identify Problem
Directed Dialog:
ID Phone
Implicit: ID Caller

Voice Conversational Model:
Troubleshooting
• So, first let’s check
your settings. What
about Airplane
Mode… do you have
that turned off? If
you need help with
that, say “tell me
more.”
• Tell me more
• Sure. You can find
Airplane Mode under
“Settings.” (call
continues)
Step 1 + Fallback

Chat Conversational Model: Troubleshooting
• Let’s check settings.
Do you have
Airplane Mode
turned off?
• IDK
• OK to check
Airplane Mode, go
to “Settings.”
Step 1 + Fallback

Error Recovery - Voice
• If the recognizer doesn’t get something,
you may have to reprompt.
– How crucial is the information you needed to
collect in order to move the conversation
forward?
• Eg: Alexa doesn’t always reprompt
– Systems can be more directive if failure
continues
“What color is that light?”
It’s… <noise>
“Sorry, what color?”
“Sorry, what color?”
<silence>
“You can say ‘red’ or press 1, ‘green’ or press 2, …. (etc)

Error Recovery - Chatbot
• Chatbots do need it!
– Do you need to do a local recovery vs. a
global recovery?
– Specific point
– Total reset
35Source: Chatbot.fail

Prompts and Responses
• Prompts imply more than choices
– Would you like chocolate or vanilla?
• Yes
• Both
• Inputs are more like text boxes

Grice’s Maxims
Maxim Description
Quantity
Be as informative as is required, but not more
informative.
Quality
Do not make statements that you believe are false or for
which you lack evidence.
Relation Be relevant.
Manner
Avoid obscure expressions
Avoid ambiguity
Be brief
Be orderly

Paul Grice
• British philosopher of language
• “Logic and Conversation,” 1975
• The maxims describe the
“default” assumptions that
listeners make about how
people talk.
• When the assumptions are
broken:
– People look for underlying
implications
– People may assume that there
is a problem with the
conversation or speaker
Source: Amazon

Grice’s Maxims: Applied
• Be as informative as is required, but not
more informative.
– What is your user really wanting to do?
• Goals and tasks
– Am I telling the user too much during this
conversation?
• Unhelpful help.
• Overly chatty bots

Source: Taco Bell Bot (That’s Nerdalicious)

• Be relevant.
– Users want to do what they want, not what
you want them to do.
• Upselling
• Intrusive apps / bots
– Maintain state

• Avoid obscure expressions
– No jargon
• Avoid ambiguity
– Ex: “This” and “that”
• Be brief
• Be orderly
– Organize information cleanly – ensure that IA
(if applicable) is understandable and relevant.

Voice
Systems
Taxonomy of CUIs
Domain
•Alexa
•Google Home
•Wildfire
•IVR
•Troubleshooting
•Manage Accounts
Technology
•Are you doing speech
recognition?
•What kind?
•What “thing” are you
talking to?
•What else are you
connected to?
•Network
•Data
•What other input
modalities are you
coordinating with?

Possibilities
Write an app (skill) for
an agent such as
Cortana / Alexa / Jibo
Use cloud APIs to add
ASR to your app / device
/ page / gadget
Download an ASR engine
and use full-featured
capabilities for onboard
robust recognition
Build your own ASR /
DNN

Distributed: Today’s Speech Agents
• Siri
• Cortana
• Google Now
• Amazon Echo (Alexa)

Alexa “Skills”
• “Alexa, ask Yelp to find me a restaurant.”
– Cortana has similar integration
• Register your skill with Amazon and
publish it

Today’s Cloud-Based Speech APIs
• Distributed speech recognition
– Collection and compression of speech is on
the device
– The language models are typically on the
network
– Can be speaker-dependent
• Trains itself on your voice and on the acoustic
environments you are in most often
– Many companies are providing APIs to use
their speech recognition

Speech API Example: Alexa Voice
Services

Possible Approaches
Cloud-Based APIs
• Easy to get started
• Lightweight
• Not much
specialized
knowledge
• Limited
Download / Embed / Roll
Your Own
• Customizable
• Can be device-specific
• More features
• Higher powered
• Will require specialized
knowledge
– Speech scientist
• Can use open source

Should I Speech-Enable X?
Desktop App / Website
• Easy to get started with
API-based ASR
• But the use case may
not be as powerful
Tablet / Mobile
• Stronger use case
• But will the network be
available for APIs?
Industrial Device
• Great use case esp. with
multimodal
• But this is harder to do
and probably will be
custom
Gadget
• Decent use case
• APIs are tailored for this
• Will they do everything
you need?
• Will the extra modality
be a plus or just a “silly
add-on?”
• Different modalities?
Car
• Safety considerations
are high here
• Need better user
interfaces & more
robust
IVR
• Touchtone can still be
good for a lot of
applications
• Speech is good for
complex call routing and
input

CUI: Past and Future
• How can chatbots learn from existing VUI
design? What makes these new interfaces
different, and how are they similar? Where
do the Alexas and Siris come into the mix?
We’ll discuss text-based vs. voice-based
conversational user interfaces, and the
landscape of Conversational User
Interfaces, now and into the future.

Iron Man 2: Marvel Studios, Paramount Pictures

Resources
• The Voice in the Machine: Building
Computers that Understand Speech –
Roberto Pieraccini
– YouTube video: “Open the Pod Bay Doors, Siri”
• Conversational design from Google:
– https://developers.google.com/actions/design/
• Alexa VUI design
– Alexa Skills Kit Voice Design Handbook”
• Association for Voice Interaction Design -
AVIxD.org
– Wiki, Brown Bags, Journal
– Looking for members and volunteers!

Thanks!
@crispinTX
crispinreedy.com
creedy@versay.com

Conversational User Interfaces, Past and Future

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Conversational User Interfaces, Past and Future

Semelhante a Conversational User Interfaces, Past and Future (20)

Mais de Crispin Reedy

Mais de Crispin Reedy (11)

Último

Último (20)

Conversational User Interfaces, Past and Future

Notas do Editor