Building WebRTC based interesting features and services . WebRTC to stream from remote machine in IOT. Details of Ramudroid a bot meant for cleaning outdoors uses webrtc stream for remote navigation .

1. Application of WebRTC
in IOT and robotics

2. “ WebRTC is a new front in the long war for an open and
unencumbered web.
— Brendan Eich, inventor of JavaScript

3. Hi ! I am Altanai
I have worked on telecom core , VOIP , Media Processing .
Author of WebRTC Integrator’s Guide .
Team
I am Deepak
Embedded developer

5. WebRTC basics

6. “browser-to-browser applications for voice
calling, video chat, and P2P file sharing without
the need of either internal or external plugins”

7. WebRTC
browser
stack

8. getUserMedia
get audio & video from
browser on secure origins (
https )
RTCPeerConnection
exchange SDP in offer
answer with peer to establish
a session
RTCDataChannel
file sharing , chat etc
WebRTC APIs
getStats
realtime monitoring
desktopCapture
screen sharing through a
extension

9. Other Audio and video APIs and features
Get User Media
● Media Stream
● media onto a Canvas
● Select input and output
media source
Video
● filters
● Resolution
● Caption ,subtitles
● Video controls
Audio
● Web Audio
● Web Speech
● Volume detection
● music visualizer

10. Communication & Collaboration for
Developers
DataChannel API
Send text to peer
Text to speech
conversion
GetUserMedia -
Audio
Speech to text
conversion on
language 1
Text
translation to
language 2

11. 11

12. Playboard
DataChannel API,
Sync array of
coordinates and
color with peer
Draw the array
values on canvas
GetUserMedia -
Video
Html5 Canvas,
Draw board,
Map drew
coordinates
in 2D array
with color val

13. 13

14. Real Time Language Translation
DataChannel API
Send text to peer
Text to speech
conversion
GetUserMedia -
Audio
Speech to text
conversion on
language 1
Text
translation to
language 2

16. WebRTC Debug Tools
chrome://webrtc-internals
opera://webrtc-internals
about:webrtc
Telephony ->RTP

17. Other interesting features ….
Augmented
Reality
Canvas Overlays
Detect Motion
WebGL and three.
js
Object Detection
headtracker
Tracker.js
SimpleCV
Recording and
transcoding
Webm record
Mp4 parser

18. Augmented Reality + Gaming + WebRTC
DataChannel API
Send info to peer
getUSerMEdia -
video
HTML5 Canvas,
Chess board
and buttons
Object tracking on
hand movement
Move button
on chessboard
on peer1
Move button on
chessboard for
peer2

20. application of
webRTC
Media Streaming &
WebRTC
back office
communications
(CRM, ERP, SCM,
FFM)
customer
care centre
HR &
recruitme
nt
Enterprise
Communicati
on &
Collaboration
Distance
Learning
Learn a
new
Language

21. IOTWebRTC &
Internet Of Things

22. IOT
stack
Altanai Bisht ( http://altanaitelecom.wordpress.com )

23. application of
webRTC in IOT
Media Streaming &
WebRTC
Surveillance
and Security
Asset
Manageme
nt
Machine to
Machine
communication
through MEdia
Streaming
data handling
in IOT
Robotics &
remote
navigation
Wearables
Smart
Cities
Power Grid
MAnageme
nt
connected
Vehicle

24. Interacting with Remote Machines
Signalling REST
Media WebRTC
GPIO control Wiring Pi
Communication Wifi

25. 25
Demo video : https://www.youtube.com/watch?
v=jrFpzN1jCR8&feature=youtu.be
Altanai Bisht ( http://altanaitelecom.wordpress.com )
Web page for control Electronic devices Hub and Electronic devices

26. Camera Access Libraries
WebRTC
Browser based
camera and
audio access
motion
RPi_Cam_Web_Int
erface
Janus
Uv4l
fswebcam
ffmpeg
flash

27. Pros and cons of different Media Capture Libraries
Motion -mjpeg
Adv:
● Easy to install and run
● Inbuilt monitoring
Disadv:
● Delay in stream capture
● Frame reload visible
Ffmpeg
Adv:
● Flexibility to change
parameters
Disadv:
● Many dependencies
● Heavier to install
and make on Rpi
h/w
Uv4l - WebRTC
Adv:
● Fast , no delay
● open codec - vp8
Disadv:
● P2p only
● Limited codec support
● Requires https to
capture from browser

28. Why is WebRTC suited for IOT ?
No plugins
No installations of flash or
any other 3rd party plugins
Royalty free
codec
VP8 , VP9 , OPUS
MIT , GPL
Rapid Support
Community support and
adoption by major browsers,
native SDKs
Javascript
Support for js is the key to
making it easily adoptable
by developers
Integrate with any signalling
method
● SIP
● XMPP
● Socketio
● websocket

29. Modes Of Communication
GSM / 3G
Range unlimited
Wifi
Range 100 m
Bluetooth Low
Energy
Range 30 m
Ethernet LAN
Wired connection

31. Big concept
Application of WebRTC in Robotics

32. Just Imagine ..
thisThis in place of

33. Ramudroid - bot to clean roads and outdoors

34. Design Blueprint
v1.0
First product blueprint of RamuDroid

35. Working
Stages of garbage collection
1. Litter comes between rotating
brushes
2. Litter is picked by brushes and
pushed upwards
3. Brushes push it towards the
tray

37. BLE control [Demo]
( Bluetooth Low Energy )
App used : Bluefruit LE connect v1.2.0

38. Components of Robot

39. Raspberry Pi
Arduino

40. GPS
Temperature sensor

41. Camera Module
Ultrasonic
80cm

42. Relay
230 v 7 Amp

43. 2 Brushes rotating clockwise and
anticlockwise

44. Counter road potholes and uneven surface

45. Web Control
Dashboard
[Demo]
Temperature Heat Sat , prec , err checksum

46. REST APIs to control Droid
[Demo]
Library : apidoc.js

47. Edge Detection [Demo]

48. Motion Tracking [Demo]
https://stemkoski.github.io/Three.js/Webcam-Motion-Detection.html

49. WebRTC call from dashboard + screen-
sharing

50. Next up ….
Garbage Segregation from wasted
collected from Bots

51. THANKS!
If you like it tweet your support to #ramudroid

53. Credits
● Presentation template by SlidesCarnival
WebRTC
● http://tools.ietf.org/wg/rtcweb/
● https://www.w3.org/2011/04/webrtc/
● https://webrtc.org/
● http://www.html5rocks.com/
● http://www.tutorialspoint.com/webrtc/index.htm
● http://caniuse.com/#search=webrtc
● http://webaudiodemos.appspot.com/
● https://github.com/webrtc/samples
IOT ref :
● http://www.instructables.com/id/Simple-and-intuitive-web-interface-for-your-
Raspbe/
Augmented Reality
● http://stemkoski.github.io/Three.js
● https://dontcallmedom.github.io
● requestAnimationFrame polyfill by Erik Möller & Paul Irish et. al.
● http://auduno.com/post/25125149521/head-tracking-with-webrtc