Apresentação feita no Google Brasil sobre como iniciar com Google Assistant, Dialogflow e interfaces de conversa em geral.

1. CONSTRUINDO PROJETOS PARA
O GOOGLE ASSISTANT
Alvaro Viebrantz
aviebrantz.com
@alvaroviebrantz

2. Alvaro Viebrantz
@alvaroviebrantz
aviebrantz.com
Google Developer Expert for IoT
Product Engineer @ Leverege
Organizador do GDG Cuiabá e DevMT

3. Um número
crescente de
dispositivos

4. COMO ISSO FUNCIONA?
Design Develop Discover

5. Google Home/Mobile device — A superfície para interagir com
o Google Assistant.
The Google Assistant — Uma conversa entre você e o Google
que te ajuda a realizar uma atividade no seu dia a dia.
Actions on Google — Como desenvolvedores podem extender
o Google Assistant (com Assistant apps)

6. COMO FALAR COM PLANTAS
🌱🛠🗣

8. How are you feeling ?
How is the weather ?
Turn on the water.
Turn on the light.

9. Ok Google, talk
to Personal
Chef
Well, it’s kind of
cold outside, so
I’d like...
Sure, here’s
Personal Chef
What are you
in the mood
for?
What protein
would you like
to use?
Speech to Text
NLP
Knowledge Graph
ML Ranking
User Profile
Text to Speech
Speech to Text
Text to Speech
...
Invoke Personal
Chef action
Parse query and
generate
response

10. How are you feeling ?
How is the weather ?
Turn on the water.
Turn on the light.

12. Are you ok ? Do you need something ?
Are you feeling a positive vibe around you ?
It’s time to drink some water.
You can take a sun bath.

13. “
Intent Matching — Relacione e categorize o que o
usuário diz com alguma intenção.
Entity Extraction — Identificar palavras chaves e
frases ditas pelo usuário.@

14. Ok Google, talk
to Personal
Chef
Well, it’s kind of
cold outside, so
I’d like...
Sure, here’s
Personal Chef
What are you
in the mood
for?
What protein
would you like
to use?
Speech to Text
NLP
Knowledge Graph
ML Ranking
User Profile
Speech to Text
Text to Speech
...
...
...
Trigger WELCOME
event
...
NLP:
Intent Matching
Entity Extraction

20. BRAÇO ROBÓTICO
🤖💪🗣

22. Mover 30 graus na base.
Colocar base em 30 graus.
Abrir garra.
Fechar garra. Pegar objeto.
Mover para frente.

23. Mover 30 graus na base.
Colocar ombro em 100 graus.
Abrir garra.
Fechar garra. Pegar objeto.
Mover para frente.

28. ● Time
● Date
● Time Period
● Number
● Cardinal
● Ordinal
● Temperature
● Speed
● Volume
● Weight
● Age
● Currency
● Country
● Location
● Language
● Artist
● Music
● ….
Entidades de sistema

29. Respostas

30. Fulfillments

32. !32

33. !33

34. PROCESSAMENTO DA CONVERSA

35. Assistant
app
{ conversation
api request }
{ conversation
api response }
Entrada do
usuário
Resposta
do app

36. PROGRAMAÇÃO
ORIENTADA A EVENTOS

37. NODEJS
PYTHON
GOLANG

38. QUALQUER COISA
( DOCKER )

40. 40

41. 41

42. 42

43. 43

44. DE UM APP PARA UMA EXPERIÊNCIA

45. Preserve e reforce sua personalidade
ao se relacionar com o usuário como
se fosse algo separado do Google
Assistant.
Tome conta do
fluxo.
Hey! This is ___
Welcome to ___
Ready to play ____
Hi! ___ here.
Hello. I’m ___
Greetings, human.
Welcome back to ___
Hey again. ___
Let’s play ___
Here’s your ___
Brought to you by ___
Hi there, ___
Let’s get started.
Ready for your ___
___, here to…
Live from ___
This is ___
What’s up, ___
and more...

47. <speak> <!-- Must be at the start of the string -->
<say-as interpret-as="characters">SSML</say-as>
<break time="3s"/>.
<audio src="https://example.com/file.mp3"></
audio>
<say-as interpret-as="cardinal">10</say-as>.
<say-as interpret-as="ordinal">10</say-as>
<say-as interpret-as="characters">10</say-as>.
<sub alias="World Wide Web">WWW</sub>.
<p><s>This is one.</s><s>This is two.</s></p>
</speak> <!-- Must be at the end of the string -->
“S S M L”
[3 second pause]
[audio file plays]
“Ten”
“Tenth”
“One Oh”
World Wide Web
[two sentences]
Reforce com SSML: Speech Synthesis Markup Language
https://developers.google.com/actions/reference/ssml

48. function welcome (app) {
return isPreviousUser(app.getUser().userId).then((userHasVisited) => {
if (userHasVisited) {
app.ask(`Welcome to Number Genie!...`, NO_INPUT_PROMPTS);
} else {
app.ask(`Hey you're back...`, NO_INPUT_PROMPTS);
}
});
}
Reforce com armazenamento e memória do seu Assistant App
developers.google.com/actions/assistant/best-practices

49. DESENHE PARA MÚLTIPLAS SUPERFÍCIES

50. AUDIO_OUTPUT SCREEN_OUTPUT
Dê suporte a múltiplas superfícies
developers.google.com/actions/assistant/surface-capabilities

51. function simpleResponse (app) {
app.ask({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number, like 42. What do you have in
mind?',
displayText: 'Howdy! I can tell you fun facts about ' +
'almost any number. What do you have in mind?'
});
}
O texto deve ser um subconjunto do audio
Suporte a voz e texto
developers.google.com/actions/assistant/responses

52. function suggestionChips (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number like 0, 42, or 100. What number do you have '
+
'in mind?',
displayText: 'Howdy! I can tell you fun facts about almost any ' +
'number. What number do you have in mind?'
})
.addSuggestions(['0', '42', '100', 'Never mind'])
);
}
Guie o usuário (chips de sugestão)
developers.google.com/actions/assistant/responses

53. function basicCard (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse('Math and prime numbers it is!')
.addBasicCard(
app.buildBasicCard(`42 is an even composite number. It ` +
`is composed of three distinct prime numbers multiplied together. It ` +
`has a total of eight divisors. 42 is an abundant number, because the ` +
`sum of its proper divisors 54 is greater than itself. To count from ` +
`1 to 42 would take you about twenty-one…`)
.setTitle('Math & prime numbers')
.addButton('Read more')
.setImage('https://example.google.com/42.png', 'Image alternate text')
)
);
}
Mostre cards básicos
developers.google.com/actions/assistant/responses

54. Usado para seleção fácil
<10 items
Usado para comparação
<30 items
Listas e Carrosséis para seleção
developers.google.com/actions/assistant/responses

55. CONHEÇA O USUÁRIO

56. Peça informações
Google Home Mobile Device
NAME Registered device user’s full name Registered device
user’s full name
DEVICE_COARSE_LOCATION Zip code and city N/A
DEVICE_PRECISE_LOCATION Coordinates and street address Coordinates
Q: "Recommend me a local
bookstore"
A: "To find bookstores near you,
I'll just need to get your zip code
from Google. Is that okay?"
conv.ask(new Permission({
context: 'To find bookstores near you',
permissions: 'DEVICE_COARSE_LOCATION',
}));
developers.google.com/actions/assistant/helpers#user_information

57. Integração transparente com Google Sign-in
Associe uma conta de usuário com seu backend com OAuth2
developers.google.com/actions/identity/

58. Account Linking
developers.google.com/actions/identity/
Multi plataforma
Google Sign In + OAuth linking
Assistant apenas
Google Sign In apenas

59. •Monte pedidos
•Use meios de pagamento
fornecidos pelo Google
•Use seu próprio processador de
pagamentos (Stripe, Braintree,
Vantiv, e mais por vir)
•Atualize o status do seu pedido
Transações com o usuário
developers.google.com/actions/identity/

62. What is the temperature
inside?
Turn off all the lights.
What lights are on in the
living room?
Make it warmer.
How many lights are on?
Smart Home
developers.google.com/actions/smarthome/
Execution path for
home automation
Dim the lights in the
living room a little bit.

63. condições
do dia a
dia do
usuário
{

64. “Eu não te entendi”

65. Quando um então chamado “erro” ocorrer
em uma conversa, ele deve ser tratado
simplesmente como um novo turno no
dialogo, mas com palavras e condições
diferentes.

66. Usuários tem
intenções e objetivos

67. COMO COMEÇAR ?

68. g.co/dev/ActionsDesign

69. developers.google.com/actions

72. OBRIGADO
Alvaro Viebrantz
aviebrantz.com
@alvaroviebrantz