These slides cover a workshop called "Having fun with Janus and WebRTC" at the virtual edition of OpenSIPS 2021. The workshop guided viewers to how they could use different features in Janus to build a WebRTC Social TV application, including how to write a new plugin in JavaScript to build a virtual remote.
08448380779 Call Girls In Friends Colony Women Seeking Men
Write a SocialTV app @ OpenSIPS 2021
1. Having fun with Janus and WebRTC!
Lorenzo Miniero
@elminiero
OpenSIPS Summit Distributed
September 13th 2021,
2. Who am I?
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus
Contacts and info
• lorenzo@meetecho.com
• https://twitter.com/elminiero
• https://www.slideshare.net/LorenzoMiniero
• https://soundcloud.com/lminiero
3. Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, ACM, etc.)
• Proudly brewed in sunny Napoli(*), Italy
5. A quick reminder on what Janus is!
Janus
General purpose, open source WebRTC server
• https://github.com/meetecho/janus-gateway
• Demos and documentation: https://janus.conf.meetecho.com
• Community: https://groups.google.com/forum/#!forum/meetecho-janus
6. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
7. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
8. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
9. Modular architecture
• The core only implements the WebRTC stack
• JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ...
• Plugins expose Janus API over different “transports”
• Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg
• “Application” logic implemented in plugins too
• Users attach to plugins via the Janus core
• The core handles the WebRTC stuff
• Plugins route/manipulate the media/data
• Plugins can be combined on client side as “bricks”
• Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
10. A ton of scenarios done today with Janus!
• SIP and RTSP gatewaying
• WebRTC-based call/contact centers
• Conferencing & collaboration
• E-learning & webinars
• Cloud platforms
• Media production
• Broadcasting & Gaming
• Identity verification
• Internet of Things
• Augmented/Virtual Reality
• ...and more!
11. How do you talk to Janus?
https://janus.conf.meetecho.com/docs/rest
12. How do you talk to Janus?
https://janus.conf.meetecho.com/docs/rest
17. A few requirements for a SocialTV demo
• An interesting scenario
• A few friends connect to the same web page to watch TV
• They can talk to each other while watching
• Any of them can change channel at any time
• Effective example of combining different plugins
• We’ll need something for the TV broadcasting...
• ... something to let them mingle ...
• ... and something for the interactive features (e.g., channel surfing)
• This workshop will guide you through the process
• Choosing plugins, combining handles, etc.
18. A few requirements for a SocialTV demo
• An interesting scenario
• A few friends connect to the same web page to watch TV
• They can talk to each other while watching
• Any of them can change channel at any time
• Effective example of combining different plugins
• We’ll need something for the TV broadcasting...
• ... something to let them mingle ...
• ... and something for the interactive features (e.g., channel surfing)
• This workshop will guide you through the process
• Choosing plugins, combining handles, etc.
19. A few requirements for a SocialTV demo
• An interesting scenario
• A few friends connect to the same web page to watch TV
• They can talk to each other while watching
• Any of them can change channel at any time
• Effective example of combining different plugins
• We’ll need something for the TV broadcasting...
• ... something to let them mingle ...
• ... and something for the interactive features (e.g., channel surfing)
• This workshop will guide you through the process
• Choosing plugins, combining handles, etc.
20. First obvious choice: VideoRoom plugin!
https://janus.conf.meetecho.com/docs/videoroom
21. Chatting with friends using the VideoRoom
• The VideoRoom plugin is an SFU (Selective Forwarding Unit)
• Publish/Subscribe approach
• Participants can send their media and subscribe to others
• Very widely used in conferencing scenarios
• Obvious choice for the friends mingling feature
• They’re basically in a “conference” with each other
• We may want to constrain it a bit compared to a traditional conference, though
• Maybe a limited number of participants? (e.g., 4)
• Capping the bandwidth is a good idea too (they’ll be thumbnails)
22. Chatting with friends using the VideoRoom
• The VideoRoom plugin is an SFU (Selective Forwarding Unit)
• Publish/Subscribe approach
• Participants can send their media and subscribe to others
• Very widely used in conferencing scenarios
• Obvious choice for the friends mingling feature
• They’re basically in a “conference” with each other
• We may want to constrain it a bit compared to a traditional conference, though
• Maybe a limited number of participants? (e.g., 4)
• Capping the bandwidth is a good idea too (they’ll be thumbnails)
23. Chatting with friends using the VideoRoom
• The VideoRoom plugin is an SFU (Selective Forwarding Unit)
• Publish/Subscribe approach
• Participants can send their media and subscribe to others
• Very widely used in conferencing scenarios
• Obvious choice for the friends mingling feature
• They’re basically in a “conference” with each other
• We may want to constrain it a bit compared to a traditional conference, though
• Maybe a limited number of participants? (e.g., 4)
• Capping the bandwidth is a good idea too (they’ll be thumbnails)
25. Broadcasting with the Streaming plugin
https://janus.conf.meetecho.com/docs/streaming
26. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers
27. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers
28. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers
29. Using the Streaming plugin for TV channels
• The Streaming plugin is an effective RTP-to-WebRTC broadcaster
• It works by creating so called “mountpoints” to identify a stream
• A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others)
• Multiple participants can subscribe to receive the same stream via WebRTC
• Easy to generate WebRTC streams using non-WebRTC tools
• Tools only need to understand RTP and support the right codecs
• Janus “wraps” RTP in WebRTC coat, but won’t transcode
• Browsers need to be able to decode the media
• Only need a single copy of the stream
• The Streaming plugin duplicates it to interested subscribers
30. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
31. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
32. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
33. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
34. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
35. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
36. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
37. Different ways of implementing TV channels
• One mountpoint per room?
• RTP source of the mountpoint must change when surfing TV
• Requires server side management of the media
• No need for channel synchronization among participants, but...
• ... can’t re-use mountpoints/channels for different rooms
• One mountpoint per channel?
• RTP source of the mountpoint never changes
• Participants change mountpoint they receive when surfing TV
• Does require channel synchronization among participants, but...
• ... allows channels re-use for different rooms
38. Mapping TV channels to mountpoints
• Each channel can be mapped to a separate mountpoint
• Independent channels and separate RTP media sources
• A cool feature called mountpoint “switching”
• If you’re watching mountpoint #1, you can “switch” to #2
• No need to create a new PeerConnection for the purpose, it’s the source that changes
• Only works if the two mountpoints were configured with the same codecs, of course
• For our SocialTV, different mountpoints can be different TV channels
• Participants start with a channel and can change dynamically
• Logic used to change channels can be delegated to a different plugin
39. Mapping TV channels to mountpoints
• Each channel can be mapped to a separate mountpoint
• Independent channels and separate RTP media sources
• A cool feature called mountpoint “switching”
• If you’re watching mountpoint #1, you can “switch” to #2
• No need to create a new PeerConnection for the purpose, it’s the source that changes
• Only works if the two mountpoints were configured with the same codecs, of course
• For our SocialTV, different mountpoints can be different TV channels
• Participants start with a channel and can change dynamically
• Logic used to change channels can be delegated to a different plugin
40. Mapping TV channels to mountpoints
• Each channel can be mapped to a separate mountpoint
• Independent channels and separate RTP media sources
• A cool feature called mountpoint “switching”
• If you’re watching mountpoint #1, you can “switch” to #2
• No need to create a new PeerConnection for the purpose, it’s the source that changes
• Only works if the two mountpoints were configured with the same codecs, of course
• For our SocialTV, different mountpoints can be different TV channels
• Participants start with a channel and can change dynamically
• Logic used to change channels can be delegated to a different plugin
41. Sample configuration for a mountpoint
rtp-sample-12: {
type = "rtp"
id = 12
description = "Sports"
audio = true
video = true
audioport = 6002
audiopt = 111
audiortpmap = "opus/48000/2"
videoport = 6004
videortcpport = 6005
videopt = 100
videortpmap = "VP8/90000"
secret = "verysecret"
}
42. Sample configuration for another mountpoint
rtp-sample-13: {
type = "rtp"
id = 13
description = "Comedy"
audio = true
video = true
audioport = 7002
audiopt = 111
audiortpmap = "opus/48000/2"
videoport = 7004
videortcpport = 7005
videopt = 100
videortpmap = "VP8/90000"
secret = "verysecret"
}
43. Implementing a shared TV remote
• The Streaming plugin can switch from channel to channel dynamically
• ... but this only applies to an individual participant
• How should we synchronize this among all of them?
• A ton of different ways to implement this
• Maybe an HTTP/WS based web application (e.g., via node.js)
• Could leverage an existing framework (e.g., Firebase)
• Why not an instant messaging protocol? (e.g., XMPP)
• WebRTC datachannels are an option too, of course!
44. Implementing a shared TV remote
• The Streaming plugin can switch from channel to channel dynamically
• ... but this only applies to an individual participant
• How should we synchronize this among all of them?
• A ton of different ways to implement this
• Maybe an HTTP/WS based web application (e.g., via node.js)
• Could leverage an existing framework (e.g., Firebase)
• Why not an instant messaging protocol? (e.g., XMPP)
• WebRTC datachannels are an option too, of course!
45. Using datachannels for our TV remote
• When looking at datachannels, many options available
• Peer-to-peer connections, without going through Janus?
• Piggybacking existing VideoRoom streams (audio/video AND data)
• Using the TextRoom instant messaging functionality
• Writing a new ad-hoc plugin (in C, Lua or JavaScript)
For the sake of this workshop, let’s write a new plugin!
• Writing one in JavaScript (Duktape plugin) is trivial and informative
• Exchanging datachannel messages is a simple logic to implement
• We can keep some server-side state too (list of channels, current channels, etc.)
46. Using datachannels for our TV remote
• When looking at datachannels, many options available
• Peer-to-peer connections, without going through Janus?
• Piggybacking existing VideoRoom streams (audio/video AND data)
• Using the TextRoom instant messaging functionality
• Writing a new ad-hoc plugin (in C, Lua or JavaScript)
For the sake of this workshop, let’s write a new plugin!
• Writing one in JavaScript (Duktape plugin) is trivial and informative
• Exchanging datachannel messages is a simple logic to implement
• We can keep some server-side state too (list of channels, current channels, etc.)
47. Writing a Janus plugin in JavaScript
https://janus.conf.meetecho.com/docs/duktape
51. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source
52. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source
53. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source
54. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source
55. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source
56. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source
57. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source
58. Putting it all together in a web application
• Three different Janus plugins used at the same time
• Custom Duktape plugin (one handle, datachannel PeerConnection)
• Streaming plugin (one handle, recvonly audio/video PeerConnection)
• VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers)
• Potential workflow for the UI
1 Prompt participant for a display name
2 Use the display name to join as a VideoRoom publisher
3 Create a datachannel with the custom Duktape plugin
4 Create subscribers for other VideoRoom participants in the same room
5 Update UI with list of channels received via datachannels
6 Subscribe to the mountpoint associated with the current channel
7 In case of channel changes, switch the mountpoint source