O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

Next-generation intelligent applications require cognitive design

Ad

Next-generation
intelligent 

applications require
cognitive design
John Whalen, Ph.D.
Chief Experience Officer
10Pearls
P...

Ad

Brief Introductions

Ad

PhD Cognitive Science
Johns Hopkins University
Cognitive Neuroscience
Linguistics
Neural Networks/ML
Vision Science
Post D...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Carregando em…3
×

Confira estes a seguir

1 de 62 Anúncio
1 de 62 Anúncio

Next-generation intelligent applications require cognitive design

Baixar para ler offline

- Learn how to "usability test" AI interactions with humans and measure success
- Understand the two distinct ways that humans construct commands to AI systems and how, using physiological measurements, you can measure the human response to the AI system responses

Description

John Whalen explores the concept of cognitive design, describing how humans structure their commands to AI systems (syntax, word usage, prosody) and how to measure human reactions to AI responses using biometrics (facial emotion recognition, heart rate, GSR). Along the way, John shares insights into how to optimally architect the customer experience.

John offers an overview of the results of an evaluation of four major AI systems (Siri, Cortana, Alexa, and Google Assistant), tested by the young and old, those new to AI systems and those that use these tools every day, native and non-native speakers, and techies and non-techies. Each were asked to interact with the systems to request facts, complex information, jokes, commands, and calendar information while the evaluators recorded their commands, the AI response, and the human’s physiological response to the AI response (facial emotion, heart rate, and GSR).

There were several intriguing findings:

- There were two distinct ways humans constructed commands for the AI systems.
- The testers’ favorite AI systems were not always the ones that performed the best in terms of giving correct answers.
- There was a distinct physiological signature associated with a positive experience.

John explains how these findings can help you determine how you should measure the success of your AI system or chatbot and suggests new ways to predict market success that go beyond AI answer accuracy.

- Learn how to "usability test" AI interactions with humans and measure success
- Understand the two distinct ways that humans construct commands to AI systems and how, using physiological measurements, you can measure the human response to the AI system responses

Description

John Whalen explores the concept of cognitive design, describing how humans structure their commands to AI systems (syntax, word usage, prosody) and how to measure human reactions to AI responses using biometrics (facial emotion recognition, heart rate, GSR). Along the way, John shares insights into how to optimally architect the customer experience.

John offers an overview of the results of an evaluation of four major AI systems (Siri, Cortana, Alexa, and Google Assistant), tested by the young and old, those new to AI systems and those that use these tools every day, native and non-native speakers, and techies and non-techies. Each were asked to interact with the systems to request facts, complex information, jokes, commands, and calendar information while the evaluators recorded their commands, the AI response, and the human’s physiological response to the AI response (facial emotion, heart rate, and GSR).

There were several intriguing findings:

- There were two distinct ways humans constructed commands for the AI systems.
- The testers’ favorite AI systems were not always the ones that performed the best in terms of giving correct answers.
- There was a distinct physiological signature associated with a positive experience.

John explains how these findings can help you determine how you should measure the success of your AI system or chatbot and suggests new ways to predict market success that go beyond AI answer accuracy.

Anúncio
Anúncio

Mais Conteúdo rRelacionado

Anúncio
Anúncio

Next-generation intelligent applications require cognitive design

  1. 1. Next-generation intelligent 
 applications require cognitive design John Whalen, Ph.D. Chief Experience Officer 10Pearls Presented at O’Reilly AI Conference, San Francisco, 2017
  2. 2. Brief Introductions
  3. 3. PhD Cognitive Science Johns Hopkins University Cognitive Neuroscience Linguistics Neural Networks/ML Vision Science Post Doc. at UCLA during dot.com boom Professor in Psychology 
 Univ. Delaware Biometrics Numerical Cognition Founder, UX Lead Brilliant Experience User Insights Digital Strategy UX / CX CXO
 10Pearls User Insights Digital Strategy AI + UX John Whalen
  4. 4. According to Google Images, I am User Experience!
  5. 5. “Intelligent Experiences” Business reimagined
 - Significant development 
 - Data driven / AI-based Experience reimagined
 - Research, psychology 
 - Innovative UX/CX +
  6. 6. An end-to-end digital experience and enterprise software application partner. Digital Experience & Enterprise Mobile Partner Supplier of the Year: Building Digital Marketplace 50 on Fire – Hottest Companies Award
  7. 7. Special thanks: Max Kalicka, Researcher Imran Aftab, CEO Jen Foster, Director
  8. 8. Agenda 1. Why AI UX is so different
 2. Designing for how people think
 3. Introducing studies
 4. Examples & results breakdown
 5. Summary & implications
  9. 9. 1. Why AI UX is so different
  10. 10. Challenge 1: What is a usability success?
  11. 11. “Okay Google, show me a waterfall.” @johnwhalen #TheAIConf
  12. 12. “Okay Google, order me a pizza.”
 
 “Here are some nearby pizza restaurants…” @johnwhalen #TheAIConf
  13. 13. Challenge 2: 
 Humanized experiences
  14. 14. “Ask Alexa to play some music.”
 “Okay, I’ll ask her.”
  15. 15. Pronouns given It It It Her Him Her @johnwhalen #TheAIConf
  16. 16. “Hey Siri, do you love me?” [ouch!]
  17. 17. 2. Design for how people think
  18. 18. Language Wayfinding Vision / Attention MemoryEmotion Decision Making Design for how people think @johnwhalen #TheAIConf
  19. 19. Language Wayfinding Vision / Attention MemoryEmotion Decision Making @johnwhalen #TheAIConf Use brain science to build intelligent experiences What words and word order were in the command? Could user understand their choices and where they are in the system? Did the AI match expectations? Did the AI generate
 a positive emotional
 response? Did the answer help to solve a problem? Did AI focus the user 
 on the answer?
  20. 20. 3. Case in point: AI experiences
  21. 21. Prompt: Chicago, second tallest building Participant:
 
 “Alexa, what is the second tallest building in Chicago?” @johnwhalen #TheAIConf
  22. 22. Task types and AI assistants studied Siri Study1
 Personal Cortana Google
 Assistant Alexa Google
 Home Alexa SiriHound Study2
 Personal+
 Business
  23. 23. Ages Younger Older
  24. 24. Experience with AI AI experience No AI experience
  25. 25. Careers Techie Non-techie
  26. 26. English fluency Native English speaker Non-native English speaker
  27. 27. Types of requests 1. Simple facts
 2. Complex facts
 3. Commands
 4. Games/Jokes
 5. Business needs
  28. 28. Captured: 1. Request uttered
 2. AI response
 3. Facial expression
 4. Heart rate
 5. GSR (sweat)
  29. 29. 4a. Study 1 Results: Personal Tasks
  30. 30. Hotel near MOMA in NYC
  31. 31. Population of country with Eiffel Tower
  32. 32. 24hr grocery store near me
  33. 33. Do aliens exist?
  34. 34. Do you like me?
  35. 35. Answer accuracy by age 63% 68% 66% 65% @johnwhalen #TheAIConf Note: Command style variance
  36. 36. Answer accuracy by assistant 76% 68% 61% 61% @johnwhalen #TheAIConf Siri CortanaGoogle
 Assistant Alexa
  37. 37. Note: The most accurate AI tools 
 are not the most preferred. Emotion matters! @johnwhalen #TheAIConf
  38. 38. Accuracy and preference by assistant 76% 61% 68% 61% @johnwhalen #TheAIConf PREFERREDACCURACY 38% 33% 5% 5% Siri CortanaGoogle
 Assistant Alexa
  39. 39. Preferred Google Assistant: “Annoying when they are being human. I don't feel anything with them.” “It is weird that they have a personality and can already tell a joke.” “Just give me the answer” “I want general information and answers fast.” @johnwhalen #TheAIConf
  40. 40. Preferred Alexa: “Alexa feels like it is addressing me back. Feels like I am interacting with a person.” “Prefer Alexa because it tells me its not sure.” “I like when it answers the question the way I worded it.” “Responses were direct. Fun to banter back and forth.” @johnwhalen #TheAIConf
  41. 41. 4b. Study 2 Results: 
 Personal + Business Tasks
  42. 42. Task types & AI assistants Google
 Home Alexa SiriHound Study2
 Personal+
 Business
  43. 43. News on Johnny Depp
  44. 44. Flight status United 753, tomorrow
  45. 45. Find 24-hour grocery store
  46. 46. Microsoft’s last closing stock price
  47. 47. When to leave, arrive Dulles airport 10pm
  48. 48. May housing starts, up or down?
  49. 49. Text Max Kalicka “Meet me on July 10”
  50. 50. Answer accuracy by assistant 77% 74% 59% 50% @johnwhalen #TheAIConf Hound Google
 Assistant Siri Alexa
  51. 51. Answer accuracy by task type 72% 58% @johnwhalen #TheAIConf Personal Tasks Business Tasks
  52. 52. Accuracy and preference by assistant 50% 74% 59% 77% @johnwhalen #TheAIConf PREFERREDACCURACY 35% 21% 21% 14% Google Assistant SiriAlexa Hound
  53. 53. Preferred Hound:  “I do love that when you ask it a question it provides you an answer in two options.” “It remembered what I was talking about.” “I liked the way it asked more than one question to get to the answer.” “It was better at the 
 back and forth.” @johnwhalen #TheAIConf
  54. 54. Biometric signature of preferred tool Heart
 Rate GSR Positive 
 Affect @johnwhalen #TheAIConf
  55. 55. Product designers:
 We can detect which AI assistant
 is preferred and if your assistant is improving over time. @johnwhalen #TheAIConf
  56. 56. 5. Summary & implications @johnwhalen #TheAIConf
  57. 57. 1. The most accurate tools are not the most preferred. 50% 74% 59% 77% @johnwhalen #TheAIConf FAVORITEACCURACY 35% 21% 21% 14% Google Assistant SiriAlexa Hound
  58. 58. 2. Biometrics can detect emotional preference. Heart
 Rate GSR Positive 
 Affect @johnwhalen #TheAIConf
  59. 59. 3. There is an opportunity for business assistants.
 72% 58% @johnwhalen #TheAIConf Personal Tasks Business Tasks
  60. 60. 4. Design for how 
 people think. 1. Conversational
 cadence
 2. Context relevance
 3. Emotional response @johnwhalen #TheAIConf Language Wayfinding Vision / Attention MemoryEmotion Decision Making
  61. 61. Thank you! John Whalen @johnwhalen linkedin.com/in/johnwhalen John Whalen Designing for How People Think USING BRAIN SCIENCE TO BUILD BETTER PRODUCTS
  62. 62. Summary of Findings 1. The most accurate tools are not the most preferred. Emotion matters!
 2. Biometrics can detect emotional preference. 
 3. There is an opportunity for business assistants.
 4. Humanize the experience. We need to design for how people think. @johnwhalen #TheAIConf

×