On the Road to WebRTC 1.0, Including VP8 | Hacker News Reader

On the Road to WebRTC 1.0, Including VP8

webkit.org

141 pointsOberstKrueger81 comments

I like WebRTC but it's a complicated mess of protocols...

I'm currently The Erlang

Yup, lots of telephony-inspired RFCs in there. If you're not already, I suggest referencing [0] as you develop, it's a great project that is also really easy to read what's happening underneath.

0 - https://github.com/pions/webrtc

Yes, pions WebRTC was my inspiration!

@bobowzki we would love to have you on Slack https://pion.ly/slack/

If there is anything I can do I would like to support your Erlang implementation! I can answer any questions you have, and would be happy to help get more eyes on your version as well.

It would be fantastic to see Go, Python, Erlang, C++ and Browsers all communicating directly without any software in the middle :)

Thanks I will do that!

Thank goodness for pions webrtc! I changed my MMO's dev version over to it a couple months ago. This is going to be tons better than the node-electron webrtc proxies running under tmux I was using. Between this, nanomsg, and BadgerDB, I'm going to be removing all of my non-golang dependencies, which is going to enable some very cool stuff.

For one thing, I now have edge servers in my architecture. This means that I can now use my ability to serialize the state of a game instance server process, then inject it into another process without any participation from the client. This will make hot code updates really slick.

That is great to hear :)

If there is anything we can do to make it better I would love to work on fixing anything that comes up, thanks for using Pion!

I'll just add https://github.com/aiortc/aiortc here too - in case anyone reading this thread is interested in a pure Python implementation.

Thanks @cretz :)

We have had a lot of churn, but I am really proud of how we split everything out. DTLS, SCTP, ICE and SRTP are a real pain but once it works it is amazing.

We are also working on https://pion.ly/knowledge-base I want to create a resource that pulls all WebRTC info together RFCS, summations and links to relevant code etc...

If anyone wants to help we would love to have you! Especially someone who has an eye for design, we have a resource but they aren't able to help often, I am always happy to chat in our Slack channel https://pion.ly/slack/

----

I still really want to tie your IFPS work together we something like https://github.com/pions/asciirtc then we can get decentralized chat that works in browsers and desktops (with a single static binary to run it locally) would be amazing

How do you handle DTLS in Erlang? DTLS implementation in OTP is missing the use_srtp extension and muxing DTLS, RTP and STUN packets on the same port.

I'm still not sure how to do the DTLS handshake. I was under they impression that I might be able to use the ssl:handshake and then downgrade again, but I realize reading your comment it will be much more difficult...

Is there a chance you'll share your Erlang DTLS implementation or challenges in the erlang mailing list? DTLS support for WebRTC and similar use cases is still missing. s.erlang.org/browse/ERL-853" target="_blank" rel="nofollow noopener noreferrer">https://bugs.erlang.org/browse/ERL-853 ng.org/pipermail/erlang-questions/2018-January/094581.html" target="_blank" rel="nofollow noopener noreferrer">http://erlang.org/pipermail/erlang-questions/2018-January/09...

I will do that. Checked the archives after your comment and it seems use_srtp support is planned. My current understanding is that for dtls-srtp only the handshake is necessary for key exchange.

This isn't as useful if webviews in iOS don't have the same apis enabled.

It is very annoying that other browsers on iOS probably wont have access to these features and updates.

If it's not enabled in web views, there's nothing stopping browsers from polyfilling it with native code.

Yes there is because you can't implement your own browser engine on iOS. Browsers are just shells for the safari engine on iPhones.

Where did you get the impression I was suggesting implementing your own browser engine?

The whole point of the discussion surrounds what is possible in web views, not implementing your own browser engine.

I've heard mixed reports on this - the big issue is technical in that you can't implement your own JIT Javascript engine, because only JavaScriptCore (running out-of-process) can write executable pages.

The big issue is that the app store rules forbid it.

> 2.5.6 Apps that browse the web must use the appropriate WebKit framework and WebKit Javascript.

There is everything stopping you. WKWebView runs in its own process and has very limited options for integrating native code.

The model I was thinking of was native code in the native app, injected polyfill that talks to the native code over a message bus. The native code can draw over the top of the web view and pass messages in. Is there anything in particular stopping that?

I believe the problem is the "message bus" is pretty slow, and has pretty limited throughput, especially for something like audio and video.

IIRC that is exactly what ericsson did, they made a iOS browser that allowed webrtc years before apple by adding functionality to webviews.

Yes. I still don't know why Apple is not supporting getUserMedia() in WkWebView which is required for the WebRTC

This radar is active for almost two years now http://www.openradar.me/33571214

I wish you could follow duplicates of radars like in github.. That issue was "resolved" by saying its a duplicate of issue 29281220, which isn't helpful at all. That issue could be even older than the one you linked.

If this is something you care about, you should file an enhancement request, even if you write that it is a duplicate of that issue yourself.

Right now, WKWebView and SFSafariViewController (and the authentication service variant, and Web.app which is used for shortcuts on the home screen) do not have WebRTC support - only MobileSafari.app does.

Apple historically considers dupes as upvotes, so be sure to indicate to which bug you suggest they dupe it if you know of one.

Have they said why they don't just use an upvoting system? Even github kind of has that with the emoji reactions.

No.

November 2016, if you use a prefix search: http://www.openradar.me/29286980

> The VP8 video codec is widely used in existing WebRTC solutions. It is now supported as a WebRTC-only video codec in Safari 12.1 on both iOS and macOS betas.

For Steve's sake! Why not add VP8 and OPUS system-wide already when they already even are built in different parts of the system?

probably because Apple doesn't have hardware acceleration for vp8. They don't want developers to use it. It'll be an inferior UX. Less battery, hotter device.

It's a bit arrogant for them to compare battery life of h264 to vp8, as they've chosen not to support hardware accelerated vp8. All Android devices have had this for years.

I'm hoping this all settles out with av1. Netflix and Amazon will support av1, so that doesn't leave Apple with much choice.

The Intel chips that Apple uses support VP8 decoding since Broadwell and encoding since Cherryview/Braswell. On the mobile side, it was their conscious decision not to support it, just like they newer supported the free audio codecs or container formats either.

Because Apple had significant stakes in MPEG-LA and VP8 and its descendants were a significant threat to its licensing revenues. Thankfully the HEVC fiasco and AV1 have forced them to accept that MPEG is heading towards irrelevance, but I don't expect them to implement support for VPx codecs any time soon. The only thing left is to wait for AV1 to mature and begin being implemented in hardware.

Honestly I don't care much about AV1, I am hardly going to own AV1-capable hardware in less than 5-10 years anyway. What I really want is OPUS to work everywhere (HTML audio tag in web browsers, message-attached audio files in messengers (Telegram), iTunes) and possibly VP8(webm)-encoded "gifs". I have converted my whole lifetime collection of MP3s (some hundreds gibs) to OPUS to save space (and put a big collection of audio books on my Android phone) and now I have to convert them back every time I want to let my wife listen to something on her iPhone, iPad or MacBook (there is VLC for this on Mac yet lack of web browser support is still a pain).

I've seen many mentions OPUS is going to be everywhere, Apple included, years passed and it still is not, not even in the fresh models that were meant to have it in hardware.

Opus works in macOS natively in QuickTime, but only if contained in a CoreAudioFormat container (.CAF). Tested on macOS Mojave, it works. They simply never implemented any kind of support for OGG.

macOS us the least of a problem. I don't even see a reason to use QuickTime when there is VLC that supports everything. The real deal are iOS and web browsers on both macOS and iOS. IIRC I've tried CAF in Chrome and Safari but neither worked.

Sadly until they implement the OGG container there isn't a lot to do about Opus on iOS. They could also gasp consider adding support for Vorbis and Theora, but I think they would rather stop making computers than do it at this point.

All I know about WebRTC is Chrome preventing sleep on my systems with "WebRTC has active peer connections". Made me ditch Chrome before it was fashionable to de-google.

So the only new feature i want from this WebRTC thingy is... an OFF button.

In addition to being annoying, WebRTC is a privacy hazard and I believe it should be off by default. I install this to make it so https://addons.mozilla.org/en-US/firefox/addon/happy-bonobo-...

isn’t webrtc by default off on all browsers? aren’t websites asking for permssion before the js is allowed access? and won’t it stay off when you deny the request?

No browser ever asked me if it can use it... and when I was using chrome, I looked for options to disable it and there were none.

Nope, the audio/video capture requires permission, but the rest should work without a permission prompt.

"but the rest should work without a permission prompt"

what "rest" are you referring to?

Connections can be made to other computers without explicit permission.

ah, i believe you are referring to the webrtc data channel. it leaks local IPs, but the severity depends on several factors, including whether you're running VPN and what you're using the VPN for, or just running behind a regular local network.

if you're running behind a regular local network then I wouldn't consider the local IP leakage as a "privacy hazard". local IPs are compromised already. everywhere. they are easy to guess. they are easy to obtain in native apps. etc.

there are issues when it comes to places where VPN access is crucial/vital. thankfully, very few VPN providers leak your IP nowadays, and with drafts such as what the poster above mentioned (https://datatracker.ietf.org/doc/draft-ietf-rtcweb-mdns-ice-...) this problem will be history soon enough.

Fortunately, Safari's WebRTC won't silently leak your private IP address or other network info. We've also proposed a new protocol extension that enables peer-to-peer pure data connections in a privacy-aware way.

https://datatracker.ietf.org/doc/draft-ietf-rtcweb-mdns-ice-...

As a user, I really appreciate how the WebKit team approaches new features in such a measured and careful way. New shiny things to use are great but I’m more than willing to wait if it helps to prevent glaring oversights. There’s no benefit to rushing these things in.

In Safari, we have UI that always allows you to see which tabs are using a connection and stop them.

Safari never prevented my machines from sleeping ;) It's what I use now as my main browser.

https://chrome.google.com/webstore/detail/webrtc-control/fjk...

I know it'll probably never happen, but I hope this somehow leads to Apple supporting .webm in Safari.

VP8 can only be contained in MKVs and WebMs files, so I do see this happening. Apple is also part of the consortium working on AV1, the successor to VP8 and VP9; it also uses the MKV and WEBM containers.

"uses the MKV and WEBM containers"

Maybe, but not exclusively

VP9: https://www.webmproject.org/vp9/mp4/ (uses 'vp09' & 'vpcC' in the stsd)

AV1: https://aomedia.org/wp-content/uploads/2018/09/AV1-ISO-Base-... (uses 'av01' & 'av1C' in the stsd)

YouTube's AV1 playlist .youtube.com/watch?v=2nXYbGmF3_Q&list=PLyqf6gJt7KuHBmeVzZteZUlNUQAVLwrZS" target="_blank" rel="nofollow noopener noreferrer">https://www.youtube.com/watch?v=2nXYbGmF3_Q&list=PLyqf6gJt7K...

uses mp4 as the container for the av1 (av01.0.05M.08), webm for the VP9 (vp9)

  ./youtube-dl --list-formats https://www.youtube.com/watch?v=2nXYbGmF3_Q 2nXYbGmF3_Q: Downloading webpage 2nXYbGmF3_Q: Downloading video info webpage Available formats for 2nXYbGmF3_Q: extension  resolution note webm       audio only DASH audio   55k , opus @ 50k, 1.43MiB webm       audio only DASH audio   70k , opus @ 70k, 1.81MiB webm       audio only DASH audio  118k , vorbis@128k, 2.99MiB audio only DASH audio  128k , m4a_dash container, mp4a.40.2@128k, 3.65MiB webm       audio only DASH audio  133k , opus @160k, 3.38MiB webm       256x144    144p  101k , webm container, vp9, 30fps, video only, 2.57MiB 256x144    144p  109k , avc1.4d400c, 30fps, video only, 1.02MiB webm       426x240    240p  205k , vp9, 30fps, video only, 2.81MiB 426x240    240p  241k , avc1.4d4015, 30fps, video only, 2.27MiB webm       640x360    360p  377k , vp9, 30fps, video only, 5.08MiB 426x240    240p  426k , av01.0.05M.08, 30fps, video only, 3.33MiB 640x360    360p  514k , avc1.4d401e, 30fps, video only, 4.66MiB webm       854x480    480p  696k , vp9, 30fps, video only, 8.29MiB 640x360    360p  751k , av01.0.05M.08, 30fps, video only, 6.04MiB 854x480    480p  989k , avc1.4d401f, 30fps, video only, 8.54MiB 854x480    480p 1175k , av01.0.05M.08, 30fps, video only, 9.86MiB webm       1280x720   720p 1282k , vp9, 30fps, video only, 12.27MiB 1280x720   720p 1675k , avc1.4d401f, 30fps, video only, 14.43MiB 1280x720   720p 2075k , av01.0.05M.08, 30fps, video only, 18.99MiB webm       1920x1080  1080p 2349k , vp9, 30fps, video only, 20.93MiB 1920x1080  1080p 2768k , avc1.640028, 30fps, video only, 23.33MiB 1920x1080  1080p 3759k , av01.0.05M.08, 30fps, video only, 39.32MiB 640x360    medium , avc1.42001E, mp4a.40.2@ 96k, 11.11MiB webm       640x360    medium , vp8.0, vorbis@128k, 16.66MiB 1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)

AVC1 is just H264[0]. FFmpeg says stream 0 is "Video: h264"

The page you linked about AV1 in mp4 says it uses MPEG-4 Part 31, which is under development and not final [1]. Until now I was not aware they were extending the MPEG-4 container standard to include more formats.

[0] https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Naming

[1] https://www.iso.org/standard/66062.html

The GP shows both avc1 and av01, the latter of which is AV1.

Yes. The post also said avc1 and av01 are both av1, when just av01 is av1.

I think you misread av1C (av1's equivalent of avcC) as avc1

VP8 is power hungry and GPUs don't have hardware support for decoding it.

Do you really want way worse battery life just so you can have VP8? That seems to be prioritizing the wrong things.

The AOM membership and the VP8 support may even be related. If Apple's assuming risk from submarine patents, etc. inherent in shipping an AV1 implementation, VP8 is something like a subset of it, so they can get that "for free".

The post talks about how iPhone 7's have worse battery life with VP8, which I'm sure is true, but it's still kinda funny because...it's Apple quasi-complaining here, and Apple who can solve the lack of acceleration, at least for future products.

I don't think they will support .webm until at least one or two generations of iPhone have hardware decoding for AV1 and/or VPx.

Apple has released software decoders before, but you might be right about waiting for hardware decoders first. They released an HEVC software decoder for devices that could get iOS 11, but had a chip older than the A9, (like the iPhone 5S and 6).

> Safari also comes with additional improvements, including better support of capture device selection, experimental support of the screen capture API, and deprecation of the WebRTC legacy API.

Looking very much forward to this. Currently my plugin-free in-browser screen-sharing tool [0] only works in FF and Chrome. The real question is will mobile browsers offer it. In general though, I am impressed with how well screen capturing over WebRTC works and so far haven't seen that many people needing a TURN server.

0 - https://github.com/cretz/myscreen.live

As someone who harassed you about VP8 support in the past [0], this is really great news. Thank you for all of your hard work. I am also really happy to see the transition to Unified Plan.

[0] https://news.ycombinator.com/item?id=14510045

Under what situation is H.264 not available in WebRTC and require the use of VP8?

Free software. The Cisco H.264 codec supplied with Firefox cannot be distributed with other software, despite being "open". From https://www.openh264.org/ , there seem to be licensing costs involved with the usage of this code, which have been waived for some uses, implying they were not waived for others:

> We will not pass on our MPEG-LA licensing costs for this module, and based on the current licensing environment, this will effectively make H.264 free for use on supported platforms.

EDIT: To clarify, such licensing requirements on H.264 may make codecs fall outside of the guidelines of software distributions allowing only free software, making them supply their browser without one.

If I remember correctly, only decoding for online video streaming doesn't require licensing fees. Cisco hits the fee cap with the products they sell already, so letting everyone use their binaries doesn't cost them anything.

Right the core problem is you have to use their binaries so Mozilla can't build it and ship their own.

Also includes lots of people w/ custom Chromium-based browsers that don't ship with these codecs (and libs like CEF don't compile with them enabled by default).

Many RTC stacks (and specifically openh264) also only support H.264 baseline profile. So you are likely to get higher quality with VP8 (subject to variations in the quality of the implementations on each endpoint, of course).

Some services use WebRTC in a "hub and spoke" model rather than true peer-to-peer, and they often require VP8 due to a desire to support low-end Android devices that don't have hardware H.264 decoding.

I am surprised to hear that. Even $50 Android Phones now has H.264 hardware decoding support, what phones are those that doesn't come with it? And likely doesn't have VP8 hardware decoding either.

VP8 and VP9 hardware decoder acceleration is common on new products nowadays?! Or is this only the case on non Apple Smartphones?

btw: Looking forward to AV1 HW decoder accelerated CPUs.

excellent news. the webkit team are really improving their webrtc offering.

i feel the next great leap for webrtc is wasm + webrtc:

https://www.google.co.uk/amp/s/bloggeek.me/webassembly-in-we...

This is super exciting, so many cool things happening here.

I haven't had a chance to grok it fully yet but someone added WASM support to pion recently [0] you will able to just write your code once in Go and should work in both places!

https://github.com/pions/webrtc/blob/master/examples/README....

wow, that's brilliant.

Is there a way to do WebRTC without revealing your IP to some signaling server?

If you know the ip adress of the peer and you can adress him directly. You dont need a signaling server in that case.

Not quite. You do need _a_ way to get the IP to the other party. How, that's up to you, the developer.

When will Safari support Opus audio codec in Ogg container?

Reply on news.ycombinator.com