P2pcf: P2P WebRTC via Cloudflare Workers | Hacker News Reader

P2pcf: P2P WebRTC via Cloudflare Workers

github.com

91 pointsrgbrgb47 comments

Neither this project, nor its predecessor are p2p. In essence both are centralized WebRTC signaling servers/services.

Sadly both WebRTC and WebSockets seem to have been designed in such a way that excludes pure peer-to-peer operation.

Author here, obviously not. The goal of the project isn't to try to solve a problem that can't be solved, it's to reduce the effort needed to set up a WebRTC signaling server. The "P2P" here is referring to the peer network that is established via the signaling.

Thank you for your very clear, no nonsense response.

I had this exact idea a few months back and wondered if anyone had done this. Nice job :)

>Sadly both WebRTC and WebSockets seem to have been designed in such a way that excludes pure peer-to-peer operation.

What do you mean by pure peer-to-peer operation?

When there's no need for a central server(s) to establish connection.

Is there any P2P network that fits that criteria? The first time you connect, you need to know at least one publicly reachable node but I fail to see how that relates to WebSocket/WebRTC. I think central server is somewhat ambiguous because the bootnode could be any node on the network you have prior knowledge of.

As I mentioned in the sibling, there's multicast/broadcast (classic multiplayer games used it) and you could do IP scanning.

I still don't understand what this has to do with WebRTC/WebSocket. You can combine, e.g., mDNS with WebRTC and build a pure peer-to-peer network.

You can't use mDNS in the browser.

The standards are carefully defined in a way that creates server dependency. WebSocket outright does not allow connections to other browsers directly. WebRTC does, but you are extremely limited in how you could do discovery. I think both are intentional.

Pure P2P node discovery is practically impossible. You can use DHT till some extent, but initial discovery will be done using some traditional peer tracker

You could do scanning and/or broadcasting.

You'd have to know the subnet at least

You will know your subnet if you are allowed to use STUN servers. Although it also hints to my original complaint about WebRTC being designed in a way that makes p2p hard for no good reason.

You knowing your subnet is not the issue. The other party needs to know it.

WebRTC needs info about where to try to connect. It doesn’t specify the signaling method which is how you get that information and share it.

If you’re already on the same subnet, use mDNS.

If you’re on different subnets it requires some kind of way to share the subnet info.

If you’re on different networks behind NAT then scanning isn’t very helpful. You’d need to scan IPs and ports because NAT randomizes the outbound ports. That’s what STUN helps you figure out. Once you’ve got that info and you build the session description structure you can send that to the other party however you want.

Public, free STUN servers exist. And people use them for all kinds of WebRTC stuff.

Neither WebRTC nor WebSockets prevent pure P2P. It sounds like your bigger issue is with NAT. IPv6 isn’t going to fix it though since the IP space will be too large to scan for peers.

> You knowing your subnet is not the issue. The other party needs to know it.

You misunderstood, knowing subnet is an option for discovery of nearby peers. If you know your subnet you can scan it in reasonable amount of time. Otherwise you'd have to scan the whole IPv4 space (or worse IPv6 space).

> WebRTC needs info about where to try to connect. It doesn’t specify the signaling method which is how you get that information and share it.

Yes.

> If you’re already on the same subnet, use mDNS.

You can't - you are in the browser.

> If you’re on different subnets it requires some kind of way to share the subnet info.

Yes, that's how connections are established with an intermediary signaling service. The p2p question is about possibility of establishing WebRTC connection without a signaling service that runs over something else and/or requires a centralized server.

I use WebRTC in non-browser applications primarily. For those systems you can use mDNS.

I have been working on WebRTC a lot lately. If you have ideas on how you’d solve this issue I’d take a stab at implementing them.

I’ve built a demo that lets you get two systems connected by copying and pasting the session description information. No third party systems required. But you still need to get your clipboard data to someone else. Which usually means SMS, Apple Messages, etc.

I had an idea to embed ICE candidates to QR codes, but its more like a toy project.

You'd literally have to scan QR codes on both devices, but that's the only option I have in mind

I think that’s an amazing idea. I tried with https://github.com/pion/offline-browser-communication but it requires standards improvements

If you are interested I would love to work together! Join the Pion slack and would love to brainstorm

I evaluated using this in prod, but came to the conclusion that R2 was a bit shaky as a database. CF has since come out with D1 SQL, DO (beware vendor lock-in), queues, and my favorite, MQTT pub-sub, which would be better options. They've also been teasing some vastly improved options for STUN/TURN; I wouldn't be surprised if more turnkey signalling popped up.

I don’t know what the consistency requirements are but workers Kv or r2 + cache might be smarter choices in terms of improving resilience and also performance by a lot (same goes for suggestions of using DO - R2 is kind of a weird choice to manage stateful rendezvous). Queues and pubsub feel like incorrect choices unless I misunderstood what they’re using the storage for.

Author here. KV did not provide the consistency guarantees needed for this to work. If you were signaling across two edge nodes, you didn't see candidates until the data was replicated, which took a long time. R2 doesn't have this problem.

This is because R2 uses Durable Objects under the hood to achieve consistency. But you'd probably get better results using Durable Objects directly.

At the time I wrote this it was prohibitively expensive.

R2 was used because it was the only database-like service CF offered at the time.

IIRC the P2pcf client polls the worker for new peers. Pubsub would let you use an ongoing websockets connection, plus all the channel management logic is built-in. The game-changer is that they've said they'll charge per-message as opposed to per-connection-second, unlike everyone else!

That’s not true though. DO/D2 are database as well (DO precedes R2 and D2 overlapped). KV is an eventually consistent database.

DO/D2 will likely perform better than R2 unless you’re storing everything in the metadata. KV will perform better than all of them although if the storage is to keep a list of peers the API as exists today won’t work well for that - you’ll have to do 1 key per connection and retrieve via a paged list - otherwise you can lose clients since KV has no mechanism to do durable updates of values stored within keys*. You could also do DO with background refresh of cache to maintain good performance while maintaining better bounds on how long a value stays in the cache.

* actually it is possible if you mediate updates to KV through a DO (with storage as the ground truth or without it and risk occasional lost writes if the CF infrastructure decides to kill and reset/relocate your DO).

Fair enough, I didn't investigate them very far, as IMO only pubsub maps to the "webrtc signalling for misers" problem neatly enough to make it worth adopting (probably -- they haven't actually said how much it will cost).

At the time I wrote this library using websockets on Cloudflare would have been prohibitively expensive compared to this polling method. Has that changed?

From my understanding, yes. I asked about the billing model on the CF Discord 6-12? months ago. I don't know how much it costs per message, or if there is a catch wrt cheap wrtc signalling, however my understanding is that you won't be paying per user or per hour etc. To be clear this is through their MQTT 5.0 service only.

Yes, in the last year Durable Objects introduced "hibernation", in which the server object can shut down with WebSocket connections still active, to be started back up when a new message arrives. This largely solves cost issues with WebSockets on Workers / DO.

Yeah this might be a better solution now for sure.

I think it'll work pretty well! I might keep something cloud portable cooking away in the background.

D1 looks like it could replace R2. Most of the tricky business in p2pcf is dealing with all the ICE details - porting the persistence from R2 to D1 might not be too bad.

can we stop calling cloudflare worker "serverless" ? There are multiple god damn servers.

I suggest you review the definition of serverless.

Serverless computing is a cloud computing execution model in which the cloud provider allocates machine resources on demand

yeah we had that some 15 years ago like Google App Engine

Yes, App Engine was an early serverless platform -- even before the term "serverless" had been coined. The technology has improved since then, though.

The cool part is that the way CloudFlare does it is really cheap

the fact that 'serverless' platforms use servers doesn't take away from the fact that the label 'serverless' has useful meaning in describing a kind of platform

you should just start re-writing the word in your head to something less upsetting since it's not going anywhere and it actually does contain information

But do you have to manage them?

Some people jokingly call that Stored Procedure in the 21st century.

and god damn clouds are in the sky!

maybe we need better names for these kinds of computer configurations.

Why oh why does this use R2 instead of CF's KC product? For just signaling? I don't get it.

I meant KV, and another comment mentions that it didn't exist at the time? I don't understand that though, since KV debuted in 2018, R2 in 2022-ish. (see: https://blog.cloudflare.com/introducing-workers-kv)

Author here. I tried KV first. It didn't work because of replication lag between edge nodes.

Sounds like a use case for Durable Objects?

Does this support screen sharing too?

Reply on news.ycombinator.com