1. The document discusses adding speech recognition and speech synthesis capabilities to web applications using the Web Speech API specifications. 2. It provides code examples for using the SpeechRecognition interface to translate spoken words to text and the SpeechSynthesis interface to output text as synthesized speech. 3. The document demonstrates how to build a simple speech-enabled question and answer application that allows the user to ask questions out loud and receives the answers as synthesized speech.

1. JavaScript
Speech
Recognition
Applicationsand maybe some other HTML5 goodness

2. Who is this guy?
core contributor
nutty about
IBM'er
@macdonst
macdonst on Github
simonmacdonald.com
PhoneGap
speech recognition

3. Why do I care about speech rec?
Here's a conversation between
two Cape Bretoners
P1: jeet?
P2: naw, jew?
P1: naw, t'rly t'eet bye.

4. And here's the translation
P1: jeet?
P1: Did you eat?
P2: naw, jew?
P2: No, did you?
P1: naw, t'rly t'eet bye.
P1: No, it's too early to eat buddy.

5. What is
speech
recognition?

6. Speech
recognition is
the process of
translating

7. the spoken
word into text.

8. The process of speech
rec includes
1. Record and digitize the audio data
2. Split data into phonemes
3. Apply the phonemes to the recognition model
4. Analyze the results against the grammar
5. Return a confidence weighted result

9. Basically...

11. So how do we
add speech
rec to our
app?

12. You may look at the W3C
Speech API Specification

13. but only Chrome on the
desktop has
implemented that spec

14. But that's okay!

15. The spec looks like this:
interface SpeechRecognition : EventTarget {
// recognition parameters
attribute SpeechGrammarList grammars;
attribute DOMString lang;
attribute boolean continuous;
attribute boolean interimResults;
attribute unsigned long maxAlternatives;
attribute DOMString serviceURI;
// methods to drive the speech interaction
void start();
void stop();
void abort();
};

16. With additional event
methods to control
behaviour:
attribute EventHandler onaudiostart;
attribute EventHandler onsoundstart;
attribute EventHandler onspeechstart;
attribute EventHandler onspeechend;
attribute EventHandler onsoundend;
attribute EventHandler onaudioend;
attribute EventHandler onresult;
attribute EventHandler onnomatch;
attribute EventHandler onerror;
attribute EventHandler onstart;
attribute EventHandler onend;

17. Let's recognize some
speech
Click to Speak
hello world
var recognition = new SpeechRecognition();
recognition.onresult = function(event) {
if (event.results.length > 0) {
var test1 = document.getElementById("test1");
test1.innerHTML = event.results[0][0].transcript;
}
};
recognition.start();

18. So that's
pretty cool...

19. ...if taking
dictation gets
you going

21. But I want to
do something
more exciting
with the

22. result

23. Let's do something a little
less trivial
Click to Speak
recognition.onresult = function(event) {
var result = event.results[0][0].transcript;
var music = document.getElementById("music");
switch(result) {
case "jazz":
music.src="jazz.mp3";
music.play();
break;
case "rock":
music.src="rock.mp3";
music.play();
break;
case "stop":
default:
music.pause();
}
};

25. Which seems
much cooler
to me

26. Let's ask the web a
question
Click to Speak
Q: what day is it today
A: Friday July 19th, 2013

27. Works pretty
good...
...but ugly!

28. Let's style our
button with
some CSS

29. +
=
<a class="speechinput">
<img src="images/mic.png">
</a>
#speechinput input {
cursor:pointer;
margin:auto;
margin:15px;
color:transparent;
background-color:transparent;
border:5px;
width:15px;
-webkit-transform: scale(3.0, 3.0);
}

30. And we'll add some color
using
by Nicholas Gallagher
Speech
Bubbles
Pure-CSS-Speech-Bubbles

31. Then pull it all
together!

32. what is steve jobs middle name
Steven Paul Jobs

33. But wait, why
am I using my
eyes like a
sucker

34. We'll output the answer
using SpeechSynthesis

35. The SpeechSynthesis
spec looks like this:
interface SpeechSynthesis {
readonly attribute boolean pending;
readonly attribute boolean speaking;
readonly attribute boolean paused;
void speak(SpeechSynthesisUtterance utterance);
void cancel();
void pause();
void resume();
SpeechSynthesisVoiceList getVoices();
};

36. The
SpeechSynthesisUtterance
spec looks like this:
interface SpeechSynthesisUtterance : EventTarget {
attribute DOMString text;
attribute DOMString lang;
attribute DOMString voiceURI;
attribute float volume;
attribute float rate;
attribute float pitch;
};

37. With additional event
methods to control
behaviour:
attribute EventHandler onstart;
attribute EventHandler onend;
attribute EventHandler onerror;
attribute EventHandler onpause;
attribute EventHandler onresume;
attribute EventHandler onmark;
attribute EventHandler onboundary;

38. who won the stanley cup this year
Chicago Blackhawks

39. Plugin repo's
SpeechRecognitionPlugin -
https://github.com/macdonst/SpeechRecognitionPlugin
SpeechSynthesisPlugin -
https://github.com/macdonst/SpeechSynthesisPlugin

40. THE END