A Practical Introduction
Anant Narayanan
May 30, 2013
FluentConf
What?
A set of technologies to enable real time communication in web pages through a
simple JavaScript API.
What can you build?
- Video and Audio Calls
- Screen Sharing
- Collaborative Editing
- Multiplayer Games
- File Sharing
How?
You need three things for this to work:
getUserMedia
var video = document.getElementById("tehvideo");
var button = document.getElementById("tehbutton");
button.addEventListener("click", function() {
navigator.mozGetUserMedia({video:true}, function(stream) {
video.mozSrcObject = stream;
video.play();
}, function(err) {
alert("Error! " + err);
});
});
getUserMedia
Integration with <video> is what makes it interesting!
Head Tracking with Headtrackr
PeerConnection
Create a P2P network channel for audio, video, and data!
High-level API, hides the grimy details of connectivity checks:
var pc = new mozPeerConnection();
pc.addStream(videoStream);
pc.onremotestreamadded = function(stream) {
document.getElementById("remotevideo").src = stream;
}
pc.createOffer(function(offer) {
pc.setLocalDescription(offer, function() {
$.post("/offer", offer, function(answer) {
pc.setRemoteDescription(answer, function() {
alert("Call established!");
});
});
}, error);
}, error);
Signaling Steps
Based on a simple Offer - Answer mechanism
createOffer,
createAnswer
setLocalDescription,
setRemoteDescription
So Many Callbacks!
pc.createOffer(function(offer) {
pc.setLocalDescription(offer, function() {
$.post("/offer", offer, function(answer) {
pc.setRemoteDescription(answer, function() {
alert("Call established!");
});
});
}, error);
}, error);
Why do we have them?
Ok, I lied
Interop? ICE success rate? (~80%)
Worry not, you have the control you need. TURN is supported!
v=0
o=Mozilla-SIPUA 8361 0 IN IP4 0.0.0.0
s=SIP Call
t=0 0
a=ice-ufrag:f34e9c4c
a=ice-pwd:639eb896a2064ffc19fee7772b875870
a=fingerprint:sha-256 4F:B9:96:14:21:06:F0:EC:1F:3E:57:FC:3A:4B:74:84:1F:35:F5:2D:85:22:5D:B6:03:AF:C6:FF:B6:A2:CC:F1
m=audio 46878 RTP/SAVPF 109 0 8 101
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
c=IN IP4 208.66.27.15
a=rtpmap:109 opus/48000/2
a=ptime:20
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv
a=candidate:0 1 UDP 2113601791 10.0.1.19 50250 typ host
a=candidate:1 1 UDP 1694236671 208.66.27.15 46878 typ srflx raddr 10.0.1.19 rport 50250
a=candidate:0 2 UDP 2113601790 10.0.1.19 61670 typ host
a=candidate:1 2 UDP 1694236670 208.66.27.15 41016 typ srflx raddr 10.0.1.19 rport 61670
m=video 33261 RTP/SAVPF 120
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
c=IN IP4 208.66.27.15
a=rtpmap:120 VP8/90000
a=sendrecv
a=candidate:0 1 UDP 2113601791 10.0.1.19 64690 typ host
a=candidate:1 1 UDP 1694236671 208.66.27.15 33261 typ srflx raddr 10.0.1.19 rport 64690
a=candidate:0 2 UDP 2113601790 10.0.1.19 49248 typ host
a=candidate:1 2 UDP 1694236670 208.66.27.15 47350 typ srflx raddr 10.0.1.19 rport 49248
Libraries, Libraries, Libraries
The spec is low level by design
Peer to Peer Data
var peer = new Peer('someid', {key: 'apikey'});
peer.on('connection', function(conn) {
conn.on('data', function(data){
console.log(data);
});
});
Easy Audio and Video Calls
Helper Libraries
Media
- VP8 for Video and Opus for Audio.
- Opus - a high quality audio codec that's great for music and real-time speech.
- You don't (neccessarily) have to worry about encoding or decoding, PeerConnection will automatically select the most appropriate codec usable on both ends.
- Integration with audio data API coming soon™
More to come...
Ready for Firefox & Chrome today. Go forth and build!