Description coming soon

1. Conversational UX Design
for Artificial Intelligence
Zhaochang He
Senior Product Designer @ VMware
zhaochangh@vmware.com
Jeremy Wilken
Staff Engineer @ VMware
wilkenj@vmware.com

2. !2
Zhaochang He
@ZhaochangHeMedium
Jeremy Wilken
Podcast Design for Voice
LinkedIn Zhaochang He LinkedIn Jeremy Wilken

3. !3
Nic Newman - The Future of Voice and the Implications for News

4. !4
Nic Newman - The Future of Voice and the Implications for News

5. !5
80,000 Alexa Skills
4,253 Google Actions up 2.5 x in past year

6. !6
Popular Alexa Skills
https://www.cnet.com/how-to/amazon-echo-most-useful-alexa-skills/
Uber
Short Bedtime Story
Ambient Noise
Flash Briefing
“Call me a uber to airport now”
“I want to hear a bedtime story”
“Play ocean sounds at 7am from Monday to Friday.”
“Give me a flash briefing from CNN”
Escape Room
“I want to play the game Escape Room”

7. !7
Nic Newman - The Future of Voice and the Implications for News
User behaviors of using smart speakers

8. !8
What’s Conversational UX?

9. !9

10. !10
Conversational UX is a user experience that combines chat, voice or
any other natural language-based technology to mimic a human
conversation. ... “A great conversational UX will feel like a natural chat
between two real people, even if one of those parties is a bot,” .

11. !11
What designer should do for Conversational UX?

12. !12
Design how a machine responds to humans

13. !13
User’s Attention Span
Short Long
WebMobileChatbotVoice
Time Spent

14. !14
Use their own words
Shorter, focused path
Natural Interactions

15. !15

16. !16
How did they communicate?
Were they successful?
Were there problems?
What did you notice?

17. !17
• Cooperation
• Shared goals
• Turn taking
• Error tolerance
• Time
• Evolving context
• Conventions
Components of Conversation

18. !18
• “Make your conversational contribution such as is
required, at the stage at which it occurs, by the
accepted purpose or direction of the talk exchange
in which you are engaged.” - Paul Grice - Logic and
Conversation
• “Do whatever is necessary to achieve the purpose
of your talk; don’t do anything that will frustrate
that purpose.” - Aloysius Martinich, Communication
and Reference
Cooperative Principle

19. !19
• Quantity - The right amount of information
• Quality - Accuracy and truthfulness
• Relation - Relevancy and context
• Manner - Briefness, logical, and unambiguous
• Politeness - Respect, positive experiences
Conversational Maxims

20. !20
• Don’t blame the user
• Reframe questions and language on reprompts
• Everything is a new turn
• Use repair techniques
• Allow users to quit gracefully
There are no ‘errors’

21. !21
“Hey Google, play Animal A to Z.”

22. !22
• DialogFlow for intentions
• Firebase for code logic
• Actions on Google SDK and console
Tools I used

23. !23
Identify a unique problem for voice AI to solve
• AI can look at a data library faster than human
• We don’t want to create content by ourselves
• Design conversations that can be answered by True/False, Yes/No
• Think about a road trip context
• Frequency: user can come back daily

24. !24
Core game logic, make it simple
Tell me an
animal name
begin with
letter “C”
Can you
spell it out?
You got 100
points!
Animal Fun
Facts

25. !25
What are the possible answers?
“Cat”
“I don’t know”
“Corn”
“@#%&$!”
“Stop”
Tell me an
animal name
begin with
letter “C”

26. !26
Design responses to bring user back on track
Great! You got 100 points!
I can give you a hint.
Sorry, that’s not a valid
animal name.
I didn’t get that. Can you
repeat?
OK. Do you want to exit
the game? You can
always resume by saying
“Resume Animal A to Z”.
“Cat”
“I don’t know”
“Corn”
“@#%&$!”
“Stop”
Tell me an
animal name
begin with
letter “C”

27. !27
What developers need? The flow chart!

28. !28
Provide a rich voices experience
Provide varies way of speaking
Engagement and thrills
Music, Sounds and Effects
Count down, “You are 5 seconds left”
Great job!
• Congratulations, you got 100 points!
• You got it, you have 100 points!
• Great! You got 300 points in total!
• Hurray! You got it. You have 100 points!
https://developers.google.com/actions/tools/sound-library/

29. !29
Multimodal? Voice first!
Parental Control on Mobile Interactive Visual
A cat is
pregnant for
about 58-65
days.
Learned 5
animal names
Successfully
spelled 4
names!
Played 30 min

30. !30
Data driven design for AI
Easy Hard
B(160), C(157), S(180) Q(3), U(2), Z(3)
Start with the letters with the larger number of animal names

31. !31
Analytics for your voice app
Track user’s conversation
ChatBase- Chatbot Analytic
Common answers
Paths
Exit point
https://supermetrics.com/blog/google-chatbase-analytics

32. !32
Modular design, create a voice component library
Tell me a [animal] name begin with letter “C”
[food]
[plant]
[transportation]
…

33. !33
Data model & prototyping
Prototyping tool without coding:
https://botsociety.io/
https://botframe.com/editor/new
https://chatfuel.com/
https://dialogflow.com

34. !34
Apply conversational UX to your product
Gmail: voice input search box

35. !35
Mobile context, easy query
Wavefront by VMware: conversational query builder
FormatQueries Axis
Legend Description
Metric
Functions
Filters
Query A
Select metrics
Advanced
Add
Add
Untitled Dashboard New Chart
Live Last 1 day1d
Line Chart
Series Name 3Series Name 2Series Name 1
0
20
40
60
80
100
6 PM5:305 PM4:30
NO DATA
Hey Wavefront, show me
the average AWS EC2 CPU
utilization.
FormatQueries Axis
Legend Description
ADD QUERY
Metric
Functions avg
Filters
Query A
aws.ec2.cpuutilization
Advanced
Untitled Dashboard New Chart
users.count
unit
percent%
x unit0
20
40
60
80
100
6 PM5:305 PM4:30
Live Last 1 day1d
Add
FormatQueries Axis
Legend Description
Metric
Functions
Filters
Query A
Select metrics
Advanced
Add
Add
Untitled Dashboard New Chart
Live Last 1 day1d
Hey Wavefront, show me the
anomalies on this chart.
users.count
unit
percent%
x unit0
20
40
60
80
100
6 PM5:305 PM4:30
FormatQueries Axis
Legend Description
ADD QUERY
Metric
Functions avg
Filters
Query A
aws.ec2.cpuutilization
Advanced
Untitled Dashboard New Chart
users.count
unit
percent%
x unit0
20
40
60
80
100
6 PM5:305 PM4:30
Live Last 1 day1d
Add
! 2 Anomalies
Disclaimer: this is theoretical demo, not a real feature

36. !36
Workshop Agenda
Group activity 0: Choose one project per group
Group activity 1: Design a starting point
Group activity 2: Design the personality for the bot
Group activity 3: Draw a scenario tree
Group activity 4: Write up scripts
Group activity 5: Role-play the bot
Group activity 6: Refine scripts (if time allows)

37. !37

38. !38
Choose a Project
Design a voice
device for car
Design a voice
device for kids
Design a
chatbot for
booking travel

39. !39
Design the Starting Point

40. !40
Design the Starting Point
• Design the Activation Cue
• Write the first introduction
• Identify the top 3 things

41. !41
10 mins
Design the Starting Point

42. !42
Design the Personality of the Bot

43. !43

44. !44
Design the Personality of the Bot
Appearance (Hair style, outfit, gender)
Voice and Tone (Pitch, fast or slow talker)
Words Selection (Modern or classic)
Behavior (Gentle, polite, cute, professional)
Branding (Color, image, animation)

45. !45
15 mins
Design the Personality of the Bot

46. !46
Draw a Scenario Tree

47. !47
Draw a Scenario Tree
Context Scenarios User Needs Questions

48. !48

49. !49
20 mins
Draw a Scenario Tree

50. !50
Write Up Scripts
What questions user will ask:
“How’s traffic today”
“How long does it take to drive to home”
“What’s the best route to home?”
Identify the Entities
For example:
<Place>, <Time>,<Traffic condition>
Prepare the Answers
“Traffic to home is heavy than usual.”
“There is light traffic in Palo Alto now.
The estimate time to home is 42
minutes.”

51. !51

52. !52
• Pick one user need
• Draw the flow chart to help user fulfill the need
• Write the answers for the questions
• Incorporate the bot personality in the scripts
Write Up Scripts

53. !53
20 mins
Write Up Scripts

54. !54
Role Play the AI Bot
• Role-play the AI Bot and test the scripts with another team
• Make sure you are NOT looking at each other when role playing
• Improve your flow chart and scripts after role playing
• Remember imitate the personality of the AI bot
20 mins

55. !55
20 mins
Role Play the AI Bot

56. !56
• Identify the primary problems in the scripts
• Strike out unnecessary parts
• Add clearer language if necessary
• Write additional scripts if you have more paths
Refine Scripts

57. !57
15 mins
Refine Scripts

58. !58
Measure the AI Design
User Intent Classification
Conversation Per Session
Resolution Rate
Usage/User Acquisition
Trust
Engagement

59. !59
Multi-players, engage all the participating parties
Who’s speaking?
Who’s turn now?
Engage the other parties

60. !60
Thank you!
Zhaochang He
Senior Product Designer @ VMware
zhaochangh@vmware.com
Jeremy Wilken
Staff Engineer @ VMware
wilkenj@vmware.com
@ZhaochangHeMedium Podcast Design for Voice
LinkedIn Zhaochang He LinkedIn Jeremy Wilken