How to create a video call application with WebRTC

I worked for a company developing a custom P2P video streaming protocol. It was amazingly hard to get working properly and testing all the quirks in different routers from different manufacturers. The combination of all the NAT strategies (I think there were 5 main ones) created quite a matrix of possible ways that things can work, and you had to implement and test them all before there was any kind of standard way to do it. We built a mock internet with dozens of consumer routers, as many as we could get our hands on. We eventually got it to work and that company exists to this day, but I think they switched to WebRTC long ago.

> The last time I did any p2p networking was back in 2002 or something when you still had to do it all manually.

> It's really much easier nowadays! zonko.chat only took me 12 or so hours to build

While it may have only taken you 12 hours to build in "real time" I'd say you've been "building" it for the last 20 years. If a newbie tried to do this they could expect to spend a few weeks or more on the project would be my guess.

True but there are some open source TURN servers you can use for that.

Not necessarily, TURN servers should only be required for strict NAT. It seems that 92%* for connections are not behind strict NATs.

*https://developers.google.com/talk/libjingle/important_conce...

True, but the problem you're mentioning is solved, for the most part. STUN/TURN servers serve this purpose, there are open source ones[0][1] and you can can even use public STUN servers[2] (as you might expect free TURN servers aren't really a thing).

Solutions like Jitsi Meet[3], whereby (formerly appear.in)[4] use WebRTC to great success and are fantastic for quick meetings.

WebRTC these days is pretty mature and ready for primetime -- TURN servers are a last resort, but they're a small price to pay for something that might be free (to you the service provider) most of the time, if signaling succeeds.

[0]: https://github.com/jselbie/stunserver

[1]: https://github.com/enobufs/stun

[2]: https://gist.github.com/mondain/b0ec1cf5f60ae726202e

[3]: https://meet.jit.si

[4]: https://whereby.com

I think the most frustrating part is that people don't know what NAT they are behind. I wish WebRTC easily told people the attributes of their NAT. Not sure if it would help production deploys, but would make learning a lot easier.

Someone contributed a really cool tool to Pion stun-nat-behavior[0] that I use a lot. It prints out NAT details. It uses the modern/correct details. I see a lot of docs that still use symmetric NAT etc.. RFC 4787 [1] suggests against all that.

[0] https://github.com/pion/stun/tree/master/cmd/stun-nat-behavi...

[1] https://tools.ietf.org/html/rfc4787#section-3

And a wrong usage of "touch and go" to boot.

for proof of concepts, twilio video was really nice, everything webrtc is abstracted away. built a demo with it in 2 weeks for a potential pivot.

Xirsys and Twilio offer affordable plug-and-play TURN/STUN options, I can recommend them both.

After hand-rolling my own setups and working with a few libraries, I have found https://mediasoup.org/ v3 to be the easiest library to use that still gives me the freedom to work with the architecture I want. This of course you're not using WebRTC for its p2p capabilities and are willing to scale via SFUs which is a common approach these days.

I am really interested in this and would love to read a technical blog post on how you built Squawk. Was it in Electron?

Especially, how do you maintain all connections so users can listen to all channels at once.

I loved the idea and interface etc. so I registered and downloaded the app. But upon launching, my corporate (ge.com) proxy blocked it:

    Not allowed to browse Newly Registered Domains category
    URL: https://app.squawk.to/

This is perhaps because your whois records are bare minimum!

    $ whois squawk.to
    Tonic whoisd V1.1
    squawk noah.ns.cloudflare.com
    squawk kim.ns.cloudflare.com

You might want to look into that.

Also, to increase adoption, perhaps give the users a link to directly use the app.squawk.to URL to use as well. I tried that on my personal device and it works as advertised.

https://stackoverflow.com/a/24879451/5472819 is a good list also, quite a few out there! Also quite a few 'WebRTC as a service' providers.

I really think it has gotten way easier in the past couple years. I am biased though, don't want to think my effort has all gone to waste :p

Any solutions where WebRTC can be transcoded to an Mpeg2 transport stream?

I recently read through a the code bases of a bunch of these, and I would add several to the list:

https://github.com/jitsi

https://github.com/pion/ion

https://github.com/peer-calls/peer-calls

https://www.irif.fr/~jch/software/sfu/

Out of those, it seems Jitsi and Mediasoup to be better as SFUs than the rest because they have what appears to be decent-looking congestion control, bitrate allocation, and support for simulcast. The rest apparently do not (at least I couldn't find anywhere in the code where it happened).

Also interesting to note is that so many of the newer ones are written in Go based on Pion. If Pion ever gains the ability to do decent congestion control (perhaps based on transport-cc like Jitsi and Mediasoup do), that could improve things for all of those.

This is an iteration on Twilio Video's example:

https://www.twilio.com/blog/2014/12/set-phasers-to-stunturn-...

https://github.com/philnash/video-chat/tree/intro

It's not just about NAT but firewalls too. I have native IPv6 but you can't reach my computer directly from anywhere.

“Chaturbate is the best video chatting product” is a catchy line. Has anyone done a comparison post?

While the video streaming isn’t that great on zoom, my experience is the audio quality piped through their custom audio thing for maxis is pretty good.

It is a truism that porn always pushes the Internet’s state of the art forward.

Any luck with Jitsi meet ? https://github.com/jitsi/jitsi-meet

Zoom getting the music audio through mic sounds like the real problem. You should be aiming to stream the audio from digital source. Then you could have the song titles overlaid on video. There's definitely licensing issues though. The instructors are probably already not using legit licenses for their classes though.

Also a lot of audio codecs are tuned towards speech and filter out high frequencies. You should pick one meant for music.

Most of the steaming platforms use HTTP Live Streaming (HLS) because it avoids all the networking NAT headaches that come with p2p connections and it handles variable quality better because each client fetches the best quality for their bandwidth. As far as I know, with WebRTC the sender degrades quality to satisfy the slowest peer.

That said, the downsides of HLS are potentially higher infrastructure costs required to transcode video to the different qualities and somewhat related is the higher latency to live. With proper tweaking you might get 2-3 seconds of latency, but it might be too much for your use case.

If there is voice interaction between the yoga instructor and their students the HLS delay will certainly be noticeable.

Isn't your usecase just a one-to-many stream, like e.g. Youtube, Twitch, Periscope, etc. all provide? That should be easier than a proper n-way meeting.

From a technological perspective, streaming with a couple of seconds delay is a world of difference from streaming with sub-second latency. You can account for network dynamics with more buffers, encode in higher quality, even using multiple passes, transcode to multiple targets etc.

One way streaming can be prerecorded and shared as video.

The debugging bit is so frustrating, I almost spend a whole week trying to find a bug with my PeerConnections only to find out that it was that the TURN Server was misconfigured (although Trickle ICE was successful). And even then just setting up a TURN server consumed a whole day.

WebRTC doesn't do everything for you; it's really just responsible for tying together ICE with media streams. Signaling is up to you to figure out. For instance, multiple clients in one room: this is part of the signaling layer and is not WebRTC's responsibility (I built this into zonko.chat if you want to see how it works though).

Closing and restarting connections is signaling layer stuff, ie your responsibility.

Echo cancellation is really supposed to be application layer and up to you as well, but I think this will probably shift to be the browser's/WebRTC's/getUserMedia's responsibility at some point.

Re. TURN: ICE is the process that works out whether a specific client needs to relay through a TURN server. The question is: do you need to implement a TURN server? The answer is: yes, you need a TURN server. If you built a P2P app that you want to work for all users, you will always need a TURN server. You can run coturn on the same box that you serve your app from. Most likely a side project will never hit the scale requiring more than a $5 digitalocean box for TURN.

And yes, it should not be a surprise that products are time consuming to build :) WebRTC is plumbing; you probably were expecting something more like Jitsi.

It depends what the product is. If you're trying to build another Zoom (which I gather from the "rooms" question), yes, it will take quite some time. For one thing, the mesh topology of P2P won't scale up beyond a handful of users, so you'll need to make it client/server. And besides time-consuming, that starts to get operationally expensive. Decoding, compositing, and encoding high-resolution video streams in real time take some processing power.

If you want to try a platform that abstracts some parts of it (such as signaling) and aims to provide an all-in-one package (compared with WebRTC which is a collection of puzzle pieces that you are responsible to put together), have a look at OpenVidu.

The team behind Kurento is working on this (I am part of it) for people who don't really care about all the intricacies of the standard(s), and just want to build a product on too of it. A single Docker container to deploy, and you're all set to write your app.

Still, this is a complex topic so there are a thousand ways this technology can be made easier to use and understand. And I agree with other comments about the issue of debugging, there is totally an empty space in the market for a comprehensive solution that can help troubleshooting when WebRTC fails.

I added an own entry here https://news.ycombinator.com/item?id=23523830

Learn more about the details: https://brie.fi/ng#help Also see https://webrtc-security.github.io/

WebRTC is end-to-end encrypted by default. There is a signaling server that helps establishing the connections between users in a room, but after that the communication is encrypted. Also those TURN and STUN servers are only required for technical reasons to get peer-to-peer working. So no content is ever passed unencrypted.

That's the difference to other services like Zoom and Jitsi, where a server in the middle is receiving the video streams unencrypted and then redistributes. Although Jitsi is adding encryption support for that as well soon.

We started a simple webrtc app in 2018. Thought it would be simple. Now two years later we are still tweaking the code and dealing with handshakes and codecs across browsers, as well as edge cases involving firewalls and what to do if someone disconnects for longer than the timeout.

Finally we had to invent workarounds for Cordova: https://mobile.twitter.com/qbixapps/status/11564841564250398...

Did anyone set up a WebRTC that was super easy and worked rock solid with just a few lines?

What are your plans to deal with the scalability issue thus far? I think that is always the biggest challenge.

Are you trying to adjust the volume of the remote audio streams? Can you change value of the `audio` element of the DOM?

I think you could also do it with the WebAudio API. If you throw up a repo would love to try and help :) having a backend makes it so much harder to deploy/maintain stuff.

There are WebRTC SDKs for Android and iOS.