Talk by Shwetank. Using getUserMedia you can now access the camera and microphone using just plain old JavaScript! In this talk, we'll take a look at the various use cases for it and what all possibilities it opens up for web applications in the future.

1. Mic, Camera,
Action!
Shwetank Dixit
Opera Software

2. about me
Web Evangelist, Web
Standards Team
Member, W3C Mobile Web
for Social Development
Group
Twitter: @shwetank
Email:
shwetankd@opera.com

3. The web is awesome!

4. but Native apps
sometimes have an edge
But that edge is getting smaller

5. We’re adding more ways
to get input from the
user

6. We’re making existing
ways easier
Like web forms

7. We want to
Stop
Writing boring JS

8. We want to
Start
Writing exciting JS

9. Mic

10. Mic
DEMO

11. Mic
Web Audio API + getUserMedia

12. getUserMedia
if (navigator.getUserMedia){
navigator.getUserMedia({audio: true}, a_success, a_error);
}

13. getUserMedia
Mic returns raw audio samples
Cannot be played natively by the audio player
Need Web Audio API to process it and then play it

14. getUserMedia
Create audio context and media stream source

15. getUserMedia
//Create audio context and media stream source
var audioContext = new AudioContext(); // Prefix it for various
browsers
// Create an AudioNode from the stream.
var mediaStreamSource =
audioContext.createMediaStreamSource( stream );

16. getUserMedia
//Create audio context and media stream source
var audioContext = new AudioContext(); // Prefix it for various
browsers
// Create an AudioNode from the stream.
var mediaStreamSource =
audioContext.createMediaStreamSource( stream );
// Connect audio loop to hear your own voice
mediaStreamSource.connect(context.destination);

17. Recorder.js
Great library to help you record mic
input using getusermedia

18. Recorder.js
buffers the raw audio samples.
converts it to format which can be
played by an <audio> control

19. Camera

20. Use camera with
getusermedia
var options = {audio: false, video: true};

21. Use camera with
getusermedia
var options = {audio: false, video: true};
navigator.getUserMedia(options, v_success, v_error);

22. Use camera with
getusermedia
// In the HTML
var video = document.querySelector('video');
....
.....
// In the JS Code
function v_success(stream){
video.src = window.URL.createObjectURL(stream) || stream;
}

23. Use camera + Video +
Canvas for new tricks
var button = document.querySelector('#button');
button.addEventListener('click',snapshot, false);
...
...
function snapshot(){
var c = document.querySelector('canvas');
var ctx = c.getContext('2d');
var cw = c.clientWidth;
var ch = c.clientHeight;
ctx.drawImage(video_element, 0, 0, cw, ch);
}

24. Demo
Notice you can click on a polaroid
and save that image.
You can create data URIs of canvases

26. Manipulating the image from the webcam

27. Once you get the
snapshot of the webcam
into canvas
you can go crazy with it

28. DEMO
Warholiser

29. Action!

30. Tracking user actions
Motion detection
gesture recognition

31. headtrackr.js
https://github.com/auduno/headtrackr/

32. headtrackr.js
http://auduno.tumblr.com/post/25125149521/head-tracking-with-
webrtc

33. Face Detection
X, Y position of face
Width, Height of face relative to canvas
headtrackr.js

34. Demo

35. Voice driven apps
Voice modulation apps
Native photo capture in web apps
Live image manipulation and effects
Video conferencing (using WebRTC)
Accessibility enhancements to apps
Possible use cases

36. Cheers!
More questions? Ask me now
or contact me at:
shwetankd@opera.com
or on twitter - @shwetank