Reflect – Multiplayer web app framework with game-style synchronization | Hacker News Reader

Reflect – Multiplayer web app framework with game-style synchronization

rocicorp.dev

462 pointsaboodman150 comments

The demo at the top of the homepage (https://reflect.net/) is a lot of fun. While I was watching, every time the puzzle got completed, people would shake their cursor in excitement, like saying, "yay we did it!!"

The first thing I did was start making the word 'FUCK' with the pieces. As soon as others cottoned on and joined in, it was a very heart warming experience.

There's a visual bug where you can hide pieces of the 'e' behind the completed portions of the 'e' - it doesn't work for other letters because they still show the outline.

Enjoy this for the next 10 minutes, because it's about to be gone. Thanks for the bug report!

Hm, good point. Thanks!

It's very fun to troll other players with, especially that middle bar piece is easy to hide. I've also seen someone play keep-away with the last piece. Just overall very funny how much you can goof off with a simple puzzle!

Brussels gets really upset.

I don't think this demo is a good demonstration of the features this library can offer. Once you grab a piece, you seem to lock it for only you to use? This means there are no conflicts to resolve, the only conflict is when two people pick a piece at the same time, from which the only resolution is to give it to the user that picked the piece first.

I'd want a demo that shows a much better example of conflicts occuring.

Are you sure about that - when I landed on that page, folks were moving pieces around but didn't recognize that they had to fill up the ALIVE text - and I was able to easily grab a piece that another user had grabbed before and place it in the text - which triggered a eureka moment among other players and we finished the puzzle...

Man, they nailed it with this page!

Very fun experience indeed. Here's a video of what you're describing: https://streamable.com/asu261

Seems to run worse on Firefox than on Chromium / Edge / Chrome. Rather choppy, mouse cursor lagging behind and laptop fans spinning up. Smooth on Chromium and friends though.

Definitely a fun demo!

Granollers!!!

People might remember this showing up once or twice before as Replicache. Reflect adds a fully managed, incredibly fast sync server.

The local-first/realtime space is getting busy these days. Replicache/Reflect is well worth checking out for the beautifully simple data model / coding model. I'm very happy with it.

One plus point compared to CRDTs is that you get to decide how you want to deal with conflicts yourself, with simple, sequential code. I'm not up to date with the latest in CRDT-land but I believe it can be complicated to add application specific conflict resolution if the built-in rules don't fit your needs.

> One plus point compared to CRDTs is that you get to decide how you want to deal with conflicts yourself, with simple, sequential code. I'm not up to date with the latest in CRDT-land but I believe it can be complicated to add application specific conflict resolution if the built-in rules don't fit your needs.

I agree wholeheartedly — we took the same approach for PowerSync with a server reconciliation architecture [1] over CRDTs, which is also mentioned elsewhere in the comments here. For applications that have a central server, I think the simplicity of this kind of architecture is very appealing.

Props to Aaron for bringing awareness to and evangelizing server reconciliation.

[1] https://www.gabrielgambetta.com/client-side-prediction-serve...

Ah thanks - related past threads:

A JavaScript framework to build offline-first but collaborative webapp - https://news.ycombinator.com/item?id=33269440 - Oct 2022 (2 comments)

Linear clone, with realtime sync and instant UI - built with Replicache - https://news.ycombinator.com/item?id=31331660 - May 2022 (3 comments)

Replicache: Easy Offline-First for Existing Applications - https://news.ycombinator.com/item?id=22173500 - Jan 2020 (22 comments)

I worked extensively on this (also implemented it on JS) about 15 years ago, to escape boredom while on my grandma's place. I wish I had open sourced something, but back then I was a young boy with no ulterior motivations :^).

Anyway, I want to comment on this:

>For example, any kind of arithmetic just works:

Of course, it is extremely trivial to set those up as idempotent operations.

>List operations just work:

Nope ... or at least, I'd have to take a closer look at your implementation, consider:

Some sort of array/list that looks like: [1, 2, 3, 4, 5]

* User A deletes the range from 2-4.

* User B deletes the range from 3-5.

* User C inserts (uh-oh) something between 3 and 4.

All updates reach the server at the same time. What is the solution to that? Timestamps? Fallback to LLW? This breaks the transactional model.

Pick any of those updates as "the winner", what do the other users see? How do you communicate this to them?

If your answer is like "we just send them the whole array again", this breaks the transactional model (x2).

Hi! Thanks for the thoughtful question.

>List operations just work:

Fair point. Will change. What I really meant here was "(Many) list operations just work" (just like above I said "all kinds of things just work".

> All updates reach the server at the same time. What is the solution to that? Timestamps? Fallback to LLW?

There are a number of issues you might be pointing out, and I'm not sure which one you mean.

In general, you can't use list indexes as identifiers for edit/delete in Reflect because they aren't stable. That's why in this example I commented to use the item itself (if atomic) or more likely a stable ID: https://i.imgur.com/IKzmf0q.png

There is also the potential issue of User C's inserts getting deleted. This seems like a problem at first, but in a realtime collaborative context, nothing can fix this. User C's insert can land just before User A or B's delete. In this case, sync would be correct to delete user C's insert, no matter the protocol. But User C might still be sad. From her perspective, she just wrote something and it got deleted. This is the nature of two humans working in the same place at the same time and potentially having different intentions. For these problems, undo and presence indicators go a long way to avoid these problems through human/social mechanisms.

>nothing can fix this

Exactly.

>For these problems, undo and presence indicators go a long way to avoid these problems through human/social mechanisms.

x1,000 to this. This guy builds. That's exactly my takeaway as well from years working on this; provide a solid (by that I mean clearly defined) deterministic algo, and solve the rest w/ UX.

If you actually look at how people use most multiplayer editing tools they tend to avoid conflicts anyway. It’s extremely painful to have two people editing the same sentence in Google Docs let alone more than that even if it sort of works. Collaboration tends to be negotiated so you end up with I-go-you-go, reviews and other patterns forming around the use of the tools. People can also see and resolve conflicts in real-time.

That said this sort of reconciliation can be extremely painful if people are working separately offline for any length of time as arbitrarily resolving many conflicts doesn’t necessarily result in coherent results. But as I understand it that’s also an issue for other ways of solving the issue as well.

Another missing piece is signalling failure. For example two people trying to fill the last slot of a limited resource. Having to infer from the state that comes back whether your operation succeeded or not is not a fun way to write multiplayer code.

This points towards a big challenge that I think gets overlooked way too often. A lot of these sync libraries/frameworks claim to solve for both real-time interaction, and local-first, sync when you reconnect. But the UX needs are wildly different.

For the former you want (as mentioned) good presence info and undo. For the latter you (sometimes) want explicitly flagged conflicts and manual resolution, like we have in git etc.

With the Replicache/Reflect model, you could actually handle both approaches. Might not be easy but the model can support it.

If a client is offline for a while and makes a bunch of edits, would rebasing many thousands of mutations not be a major performance issue? (For example: if complexity is O(n) per edit, as is often the case with string insertions, the process might hang for a long time.)

It seems to me that maybe saying that CRDTs are "good for one particular problem," whereas Reflect can do anything, is a bit misleading? Reflect is eventually consistent, but if you want e.g. string mutations to make sense, you still need to de facto model your text as a CRDT. (Unique IDs per letter, tombstones, etc.) Otherwise, you'll end up with garbled text.

I didn't mean to say that Reflect can do _anything_. We do think it's a general model that ends up working well for very many things in practice.

You don’t even need to involve deletes; if all of these happen “simultaneously”:

  - client A appends “a” to list L
  - client B appends “b” to list L
  - client C appends “c” to list L

the server will arbitrarily apply the appends in some order, and then send the result back to the clients. But, each of the clients has applied its append locally and is showing those results until it gets different information from the server, so the next time step might be

  - client A thinks L is [“a”]
  - client B thinks L is [“b”]
  - client C thinks L is [“c”]
  - server sends state event to clients saying that L is [“c”,”b”,”a”]

Then when the clients receive the state from the server, I guess they all discard their pending mutations and use the state from the server as their state of the world. But what if each of the three appends results in that client winning the game if their mutation gets applied first? Do all the clients display “you win” for 300ms while waiting for the server’s update?

We have the "game-winning" scenario on the reflect.net homepage right now: https://reflect.net/. It's a puzzle and when the last piece is placed, it shows a celebration and the puzzle resets moments later. But we only want that logic to run once. We can't do this client-side because there are many people and we'd end up having to choose a leader or something. This is the same exact situation as wanting to show in the UI who won (who placed the last piece) and only wanting to do that once.

Reflect has first-class support for this kind of situation by allowing mutators to run special code server-side. I talk about this a bit here:

https://rocicorp.dev/blog/ready-player-two#server-authority

You can use the same mechanism to have a branch in the code that places the piece that only runs server-side and chooses the winner.

You modified the operations, I specifically chose a scenario where User C (in my example) introduces an undecidable situation. Appends are trivial to solve.

>If your answer is like "we just send them the whole array again", this breaks the transactional model (x2).

You also did that, whoops.

True. I was thinking of a simpler criticism that, if it held, would also make Reflect less useful. aboodman replied in the sibling that (of course) they thought of that, so my criticism doesn't apply.

Most of these systems do deletes with tombstones. Then you can safely insert after/inbetween deleted items.

Is the solution to that,

a) [1, 5]

b) [1, <new element>, 5]

By your example, 5 was marked as deleted, so it would just be [1, <new element>].

Most likely this would be implemented with a single mutator for each element. But if you implemented a "range delete", then it would depend on what the order of arrival was. Notably, the server has to sequence actions somehow, so there isn't going to be an issue of "all actions arriving at the same time". If C arrived last in that case, the new element would be there. But if C arrived first, you'd just be left with [1].

Whoops, you're right,

Is the solution to that,

a) [1]

b) [1, <new element>]

>Notably, the server has to sequence actions somehow [...]

Yes, that's my whole point, you have to fallback to something like LLW, and then it's over for the transactional model :P.

Edit: Btw, I'm not trying to be the snarky, pessimistic dude. This kind of models are really interesting and I love working with them. I am just trying to illustrate how things could go awry without much complexity involved.

No, the server is going to process the actions in a single queue. So again, it's whatever action arrives first.

Then it's LLW and you're no longer playing this game.

I don't know what LLW means, but you seem fixated on the acronym. It's just first come, first serve transaction ordering from the world I come from.

Perhaps it's a typo/braino for LWW, Last Write Wins.

It is! Thanks!

In a collaborative application the server could resolve the three operations in any order. The three users observe the result, realize that they are operating on the same data at the same moment and messed with the state. Then they fix it. Think about editing a document.

If they can't see each other working on the data together they could be surprised but they could eventually fix the data to the desired state.

If they don't check the result or can't check the result, the state won't be OK. However it's not what probably happens in interactive apps like collaborative editing or games.

Hey hacker news! I'm one of the people behind this, happy to answer any questions.

Aaron is humble, so I'll do it for him! He created Greasemonkey and has had a hand in several groundbreaking browser innovations over the years.

Having worked with Aaron on a few things many years ago, I can confirm this as well!

Congrats! I've been watching this space for a while, having built a couple multiplayer sync systems in the past in private codebases, including a "redux-pubsub" library with rebasing and server canonicity that is (IIUC?) TCR-like. There's a lot to like about this model, and I find the linked article quite clear - thank you for writing and releasing this!

1. You wrote "For example, schema validation and migrations just sort of fall out of the design for free." - very curious to read about what you've found to work well for migrations! I feel like there's a lot of nice stuff you can build here (tracking schema version as part of the doc, pushing migration functions into clients, and then clients can live-update) but I never got the chance to build that.

2. Do you have a recommendation for use-cases that involve substantial shared text editing in a TCR system? I'd usually default to Yjs and Tiptap/Prosemirror here (and am watching Automerge Prosemirror with interest). The best idea I've come up with is running two data stores in parallel: a CRDT doc that is a flat key/value identifying a set of text docs keyed by UUID, and a TCR doc representing the data, which occasionally mentions CRDT text UUIDs.

1. I hope to write about this soon! We've developed something we really like for Reflect.

2. This is what I want to work on next. I am similarly intrigued by the Prosemirror model. It seems like a good match.

What does this mean for Replicache development, is the client-side codebase mostly shared and can expect continued updates, or more likely to replace the primary focus for you?

The clients are almost entirely shared. We will continue to develop Replicache.

https://x.com/replicache/status/1714684061589877235?s=20

Thank you! Really like Replicache though only used it for hobby projects so far, and enjoyed also the backend side of it (minimal and entirely in supabase stored functions, probably not prod ready). Hope the new approach is a great success for you!

Curious what your thoughts are on ElectricSQL?

It seems really interesting. The key thing to understand is that there are deep architectural choices made in each of these systems that really influence what you can do. There is "mechanical sympathy" with certain kinds of applications.

ElectricSQL is a distributed database. It's not going to run anywhere near 60fps with tons of users, because it's not running in memory on the server like Reflect/PartyKit/Liveblocks do. OTOH it will allow you to filter/query interact with much more data at the same time than Reflect/PartyKit/Liveblocks do – Reflect has a limit currently of 50MB per room and it's not practical to have dozens of rooms open at one time to get around this.

Systems in the Reflect room/document based model are well suited for applications where you primarily interact with one "document" at a time and you want that to be as realtime as possible. Figma is the canonical example. You want the entire document to move together at 60 FPS, completely fluidly. It's not going to be possible to do this well in the ElectricSQL model IMO. Spreadsheets, presentations and documents are other examples. Systems in the ElectricSQL model are more suited for applications where you are interacting with lots of documents at the same time. Think CRMs, bug trackers, etc.

Of course real applications are messy. Even document editors always have a dashboard where you can at least see all your documents at once. And CRMs of course have a detail view that looks like a document editor. Both types of systems are going to track toward the other to support the needs of real applications.

HTH!

What is the maximum number of users that can be concurrently in a room?

There's no max enforced. Right now, it will fall apart pretty quick around 25-50, depending on how much work is going on.

For the GA, we plan to implement a scheme that will allow the room to support up to ~100 concurrent active users (actually doing things) and thousands just watching.

This is a pedantic comment, but the terminology is confusing. Games have "players" so their sync systems are called "multiplayer," but regular software has "users" so their sync systems should be called "multiuser." The page reads strangely mixing the terms "user" with "multiplayer."

Maybe. But user is like a consumer. So HN is multiuser, but seems weird to call HN multi-player. Live concurrent interactions feels like something more than multiuser. Just my 2-cents, but multiplayer as in multiple users acting and interacting is a good use of the term multiplayer.

Let me guess: you are not regularly playing multiplayer computer games?

I totally feel hgs3's confusion when reading the discussion here or that website. I always confuse the use cases - "are they now talking about computer games, or are they talking about the broader scope of any kind of software in which multiple users can act concurrently within a shared environment?".

I think this confusion would be much less severe if I hadn't been an avid gamer for decades and thus had a very specific idea of the meaning of the term "multiplayer".

Its logic is based on the logic that's been in "multiplayer games" for decades.

On the other hand all web software is "multiuser", which doesn't tell you anything.

That’s just the terminology these days. Multiplayer == “live” multi-user including visualizing what other users are doing at every moment.

As someone who's casually been eyeing up CRDTs for two years I've wondered continually what the story for authorization is, and this article seems to suggest (in line with my own understanding) that a CRDT library on its own like Y.js can't really handle applications where mutations need to be checked for authorization because it lacks a central authority (i.e., the server) while Reflect assumes that a server will be mediating client interactions. Is this all correct?

"Can't" is a strong word. You can get auth with a CRDT with effort, for example you can put a server in the middle of everything and have the server reverse any changes it sees which are unauthorized.

This ends up being a lot of work to maintain and easy to break as the application gets bigger, and it also defeats some of the benefits of the CRDT in the first place (now the server has to mediate everything and you can't have peer-to-peer sync).

Also if there are side-effects of any actions which require auth which aren't undoable, then that gets more complicated.

If you already have a server in the middle, it's a lot simpler to just use a protocol that allows the server to reject messages in the first place.

Any thoughts on this approach? (From the Local-First Berlin meetup in June) https://m.youtube.com/watch?v=pBvGeU7bL5A

I think it sounds really cool. I love the p2p side of local-first. Lot to figure out to make it practical, but very glad people are working on that.

CRDTs can still involve a server, just not necessarily in the middle of everything. Consider you could have connections authenticated by the server, for one.

E.g., someone connects peer to peer to you and claims some privilege etc., you could use the server to verify their claim.

Also, I don't think CRDT necessarily implies peer to peer, as you can use a central server for message passing, but keep the CRDT model for resolving the current state on both server and client.

One way to do authorization is to sign each operation/message and then verify the signature to match a public key in an access control list. This also enables CRDTs to work in a peer to peer context.

But as aboodman says in a sibling comment, if there’s a server as an authority, it can simply reject messages from unauthorized clients.

How does one handle an upgrade to mutators? If a client is running old code then the operation will differ to the server. Obvious answer would be to version them independently: `increment_v1`, `increment_v2`, but wondering if there is a better answer?

At the moment there is no better answer than do not remove your old mutators until you know there are no more clients out there that might have pending changes.

Right now that window is pretty small because we haven't turned on persistence yet.

Looks great, congrats on launching. Is this in a similar vein to https://partykit.io/?

Yes they are in the same space. The key difference is in how opinionated each is.

PartyKit is extremely unopinionated. It's essentially lightweight javascript server, that launches fast and autoscales (I don't say this as as a bad thing, it's a useful primitive). Most people seem to run yjs in PartyKit, but you can also run automerge or even Replicache – my company's other project.

Reflect is entirely focused on providing the best possible multiplayer experience. We make a lot of choices up and down the stack to tightly integrate everything so that multiplayer just works and you can focus on building your app.

That's the plan anyway :).

Thanks for taking the time to answer! I've been trying to keep up with the rise of more collaborative tooling, so I appreciate the comparison. Will have to try out reflect sometime on a future project :)

For reference, this strategy is called "deterministic lockstep" in the gamedev world, and is used especially often in games which have many entities which need their state synced. The canonical example is rtses, which pretty much all use it.

Specifically "Deterministic Rollback", as the clients don't wait for the server's response before displaying the change locally.

For an in-depth explanation, see this excellent GDC talk on its implementation in Mortal Kombat and Injustice 2: https://youtu.be/7jb0FOcImdg

Hey wheybags, this is not deterministic lockstep.

That is an algorithm for peer-to-peer games, where each peer waits for the input from all other players before advancing the game simulation. The deterministic part is because players share inputs (not simulation results) AND the simulation is deterministic for the same inputs. The lockstep part is because all clients advance at a coordinated pace. The Age of Empires series use this approach, and that’s why units don’t move immediately when you click. Starcraft uses this too, but it has some tricks to smooth the gameplay experience. Both are peer-to-peer with no single “server”.

In the case of Reflect, we have a server-authorative simulation (not peer-to-peer). Clients send their inputs to the server, but they do not wait for the result, they instead predict the result locally without confirmation from the server. The server also rolls back time and then replays inputs to compensate for individual client latency. And the client corrects/reconciles their local simulation once the server sends a simulation result that included one of their inputs.

The keywords for this algorithm are:

- Server-authoritative

- Predicted

- Lag compensated (simulation rollback / server rewind / input replay)

- Prediction reconciliation (local simulation rollback / local input replay / misprediction smoothing)

I’m not sure if Reflect has it, but client-side interpolation is also a common feature in FPS games, where upon receiving a world update, the client will tween entities to their new positions/rotations over a fixed interval (such as 0.1s). This allows you to send updates to clients at only 10Hz but have entities move smoothly (without needing the client to locally simulate the physics of every entity).

There is only one authority, the server, so determinism isn’t super important. However, it is nice to have for client predictions to reduce the occurrences of mispredictions. Mispredictions occur when the server state did not advance the way the client predicted. These can happen when:

(1) Another client’s input changed the world state in an important way,

(2) The simulation is not deterministic (e.g. the random number generation is not synced).

In FPS games, (1) is impossible to eliminate. These mispredictions are often smoothed using interpolation. (2) should be minimised, but may actually be desirable. For example, Counter Strike does not sync the RNG for bullet spread randomness, to prevent “nospread” cheat programs from predicting where each bullet will go and instantly adjusting the player’s aim so the bullet lines up perfectly.

https://www.gabrielgambetta.com/client-side-prediction-live-...

https://developer.valvesoftware.com/wiki/Latency_Compensatin...

https://developer.valvesoftware.com/wiki/Source_Multiplayer_...

Yes, I know there's some differences in what you're doing here from classic deterministic lockstep, but the fundamental concept of syncing a queue of inputs from which the game state can be derived, instead of syncing the game state itself is the same, which is why I mentioned it. Also, while deterministic lockstep can be used peer to peer, it is not fundamentally a peer to peer model. Factorio for example, uses server authoritative deterministic lockstep, and has lag compensation (but only for certain actions like movement).

All multiplayer games sync a queue of messages to derive game state, that’s inherent in networking.

Deterministic lockstep means that the speed of the simulation is dependent on the slowest member as everyone waits to get all the inputs before advancing a frame. You’ll get literally pauses and jank if someone is on a poor connection. That’s not the case here.

The difference with this model of networking is what is synced in the lockstep case the simulation being deterministic means you only need to send inputs to keep the game in sync so you vastly reduce network traffic. In this case it’s not necessarily deterministic and the server sends back the new state to the clients after determining what it is as it’s the authority on what that state is. Clients can predict ahead by making the change locally but must correct mispredictions. In the deterministic case there can be no mispredictions and it’s a pain in the butt to ensure that your game is actually deterministic and resolving sync issues is a real pain.

Factorio sounds like it uses a deterministic model that doesn’t wait to run in lockstep and instead reconciles errors that causes. The typical way of doing that is to rollback to a prior state where the simulation diverged due to missing inputs and redo the following ticks. That’s probably prohibitively expensive given the scale of Factorio versus something simpler like a fighting game so I assume there is a hybrid model and some extra info is sent by an authoritative source that can be checked locally.

FWIW, I don't think this is deterministic lockstep for the key reason that Reflect is not deterministic:

https://i.imgur.com/q9FnN4R.png

The whole idea of DL as I understand it is to take advantage of determinism to reduce bandwidth. But Reflect doesn't do this.

I think this is a feature for developers. It's nice to be able to have the server just do whatever it wants for whatever reason it wants. It also means you can use normal JS, and don't need to try and enforce determinism somehow.

There are a lot of different names for these protocols floating around. Server Reconciliation is the one I found that best describes what Reflect does: https://www.gabrielgambetta.com/client-side-prediction-serve...

But in the end I felt like since we're going to continue to evolve this and have made specific choices, it made sense to give our version its own name so that we can refer to it.

Reflect is awesome. We're currently using an alpha version in production, and I've been really happy with the system as well as Aaron and team!

Happy to answer any questions as a customer :)

Besides trystero which is great but depends on things I cannot get into our company, anything open source? I cannot get this into our company either as it's vital infra and as such needs to be replaceable with open source at will. We have that as a rule. The only things that are exceptions are necessary evil; payments & cloudflare (bot-fight specifically).

While I do love the simplicity of this approach on the surface. Some of the complexity gets brushed under the rug.

Firstly is rebasing these operations in this style of system takes a bigger hit on performance than I expected in my experience. Sure, managing counters and doing integer math can consistently run at 120FPS but more complex operations can really bog down the system when a lot of users are interacting with it once.

The second challenge is that when operations fail (e.g. for authorization) undoing that operation isn't always straightforward. You either pop the operation and replay ops from a earlier snapshot (which necessitates storing all ops in order), writing an inverse operation manually, or having the server send the entire state back to rebase your unconfirmed ops on which can also end being expensive.

In general, the challenges of this approach are true for all operation-based CRDTs too. State-based approaches are a good deal simpler just tend to over-send data over the network.

Hey Matlin, it's true that this approach is harder to implement, but that's a one-time cost, and it's one we take on for our users and they don't have to worry about.

Also just as an FYI we do have this running live at https://reflect.net and https://hello.reflect.net/examples at 120 FPS.

We are targeting 100 concurrent users at 120 FPS. Not completely there, but it is certainly achievable, and we will scale back framerate when we need to.

So are you using fps management just for scaling purposes or also as a way of ensuring fairness? The reason I ask is because games like Forza have had to resort to setting all online players' frame rates to be the same regardless of their machine's capabilities due to the way the in-game physics work. PC players on high end hardware would consistently get better lap times than Xbox users because of it.

Maybe this is a bit out of scope of what the goal is for this project but just thought I would bring it up.

Huh! That's interesting. I don't think this applies to the type of collaborative apps (ie figma) we are targeting, but it's fascinating to know. Thanks!

Can you share how y'all made it fast? Obviously it can be made fast because multiplayer online games are living proof but I'm curious what it takes to get there or what's missing in my mental model.

Not really any magic, just a lot of careful engineering.

What's your opinion on using this library as a base for (eg) a simplified "world-of-warcraft" clone?

I've done a lot of thinking (but not coding) around "simply replicating state" being the right answer to interconnected apps.

The degenerate case is to model everything as a git transaction log, but be able to run it "faster" than really using git. It's very intriguing to hear your discussion of "branching, sequentialization, etc..." because that seems like the right way to go. ( edit: and implicit _physics_ vs. state transfer in some cases... it's cheaper to send nuke@[x,y,z] instead of the literal calculations and changes to state that single action may imply ... physics in games maybe being gravity and ballistics, but 'business-physics' might be update/select/modify, etc. )

There's a few interesting optimizations/tradeoffs in some of the "real" multiplayer networking libraries: https://torque-3d.readthedocs.io/en/latest/script/network.ht...

...eg: Unguaranteed, Guaranteed, Guaranteed Order, Most Recent State, Guaranteed Quickest (and each client has it's own "camera" / perspective against the "scene", where +180 might be "need it now b/c I'm looking at it", but -180 might be "meh, need close-to-the-latest-state before I turn around and look at it").

I'm rooting for you b/c "multiplayer + physics" seems like the right answer (eg: the ghost of couchdb), but no one seems to have (yet) cracked the semantic code as to how to describe "as fast as local, but synchronized with $THESE tradeoffs and $EXCEPTIONS".

Wow this looks incredible. I had the most fun I've had on a computer in a long time this last weekend using Partykit.io to build https://is-sf-back.cconeill.partykit.dev/ and this looks so much smoother.

Going to have to check this out.

When will it finally be possible to use UDP protocol within browsers? TCP got way too big overhead. A multiplayer engine in which you send 30 messages per second for 128 users is what i need. Not possible with websockets withput having super computers.

WebRTC makes this possible, but server implementations are quite difficult to set up. I recommend you check out Pion (Golang) as a good implementation that's also easy to deploy and integrate with Node (presumably your game server is implemented in JS).

Why does TCP have so much overhead? 30 messages that you don't batch is 30 packets either with TCP or UDP.

TCP got a much higher ping delay. For real fps like multiplayer experience you definitely need UDP. Packet loss should be ignored, not retried.

This isn’t as true anymore but it’s definitely best practice to build on UDP. Games built on UDP normally have several different policies including the ability to send messages that guarantee delivery even if packets get lost.

HTTP3 is your friend here

Looks great for a hobby project.

Has anyone done the cost + risk assessment of building a for-profit product on top of this? Would love to know, as I am working on a web IDE with collaboration. There’s also the matter of obscuring client data from 3rd and even 1st party.

If what you're building on top of leaks into other parts of your codebase, can't be isolated and can't easily be swapped out, I'd say avoid building a product on top of it.

A risk assessment? Just go and try it out in a prototype and see if you can make money with it. You can always replace it once successful.

Don’t call it multiplayer, call it multiuser or concurrent user. Player assumes it’s a game. It’s not a game. It’s very very real. We are the agents. Cool project. Collaboration is hard. Synchronicity even harder. This makes steel become butter.

I like multiplayer because it connotes games and games have worked so hard at this and done so well. Our goal is to be as good as games.

But I get your argument too. Naming is hard.

I think multiplayer is a good name for a different reason. I think it’s a clear and accessible metaphor for how people will interact, i.e. by concurrently working toward a shared goal with a coherent shared state.

I don’t think it trivializes the goal or the work being done, it just distinguishes it from parallel non-coordinated use.

Also, players are not just for games; the Bard had some thoughts on that. ;)

Two hardest things, Naming and Time synchronization. Thanks for building this.

Very elegant solution, and neat demo page!

I'm curious what trade-offs you looked at regarding sending state vs mutations from the server back to clients. As the server is authoritative, I imagine it could also send mutations to the clients in the order in which the server has determined parallel operations need to be resolved (and adjust mutations as needed to resolve conflicts or permissions issues). Clients would then need to rebase any pending local mutations, so that would involve more logic vs the data sent for mutations most likely being smaller than the "full" state of certain objects being updated?

Very allagmatic. Instead of storing & transferring structures transfer operations.

Iw as wondering what kind of checkingpoint or snapshotting there might be. Such that one doesn't have to "replay-the-world" if something goes awry (localstack persistence did/l (does?) this, storing state by just recording/replaying API calls). It sort of seems like having a globally accepted state is enough & makes sense, but it still seems like work to rollback & do again, still seems to require snapshotting.

Something like this feels like a 100% fit for having immutable data structures. Where you can snapshot the world very very quickly/at low cost.

Reflect is indeed built on immutable data structures internally, and it’s part of what makes it performant.

Figma uses a similar solution for their realtime feature. They have a famous article that explains why CRDTs aren’t necessary when you have a central server authority. https://www.figma.com/blog/how-figmas-multiplayer-technology...

I implemented the same solution with Redux actions and Cloudflare Durable Workers. https://emojis.cadell.dev

Any idea how this compares to Supabase's Realtime & Presence libraries?

https://supabase.com/docs/guides/realtime

We'll post something soon comparing all of the options (in our opinion). But as I understand it:

- Supabase Broadcast: one client sends every other client a one-time message

- Supabase Presence: one client tells all other clients that its own state has changed. This is similar to above, except that server keeps track of all the last-state for each client so that new members of the channel can get up to date.

- Supabase Realtime: server tells all clients when server state changes (so you will not get optimistic client-side changes this way)

You forgot to compare them :)

Supabase’s real-time stuff lets various parts of your system send messages about changes to each other. But when it comes to realtime editing, that’s the easy part!

Reflect is a much higher level abstraction that handles the hard parts:

- maintaining a cache on the device (this alone is annoying hard to get right)

- queuing updates to send to the server reliably

- handling conflicts between different users while preserving the intent of each user, by rebasing histories locally, and choosing the winner on the server

One consequence of this approach is that your mutators will need to be versioned, and you can pretty much never delete or change them once they're live. (Please correct me if I'm wrong).

This looks great! I'm curious what the recommended use cases are for something like this. Is it worth adding the additional complexity for complex crud-y saas? I know Linear is built on this local-first model but from seeing their presentations on the challenges they've had scaling it, I'm left wondering if it makes sense for something like a full-featured CRM with hundreds of tables. Is Reflect/Replicache suited for that kind of use case?

My 2c as a game dev is that you really want to know the cases where you need real-time collaboration. Otherwise the still collaborative but slower updating techniques people normally use are perfectly fine. Even in something like an MMO parts of the game will use real-time game servers and others might just use a REST API.

The other side of it is to make sure that real-time collab is something users actually want and use. There’s a lot of froth at the moment about real-time but IMO and IME it has a much smaller set of useful cases than most people actually think. But where it is useful, it is a step up for collaboration.

For example with Google Docs I find myself and our broader team mostly solo write but collaboratively review and edit. Where we collaboratively write its carefully structured so people aren’t stomping on each other and often done with someone acting as a mediator.

We have an old demo of our Linear clone using Replicache at https://repliear.herokuapp.com/

The main value prop of CRDTs is they can be used without a central server (for example, in p2p networking, or when offline.) If you're choosing between a CRDT or something else like this for your centralized server, you're probably not thinking about things properly.

Edit: I have no idea why this was downvoted, it's true

> If you're choosing between a CRDT or something else like this for your centralized server, you're probably not thinking about things properly.

I think that’s the point of comparing this to CRDTs - to show that there may be a better tool for the job if you’ve been considering CRDT because it’s seen some recent popular exposure

So comparing this with Replicache, basically same client but with a real-time managed server?

Frameworks for hard problems like this seem like such a good public service. Is there much low-hanging fruit in this area -- common concurrency problems where people are likely to roll their own solutions because it's hard to find a preexisting framework?

I mean…how many things we gonna call Reflect? Signed, the former founder of reflect.io.

A lot. - Signed not the creator of the word reflect.

I need a simple explanation like i am 6 years old because i have a mind of a 6 years old.

This is a hosted service that makes it much easier for you add multiplayer real-time collaboration (as seen in Google Docs and Figma) to your own web application.

Not at all any expert in this space, so maybe a dumb question, but how do you model plain text data for something like a collaborative markdown editor with things like this or CRDTs?

What’s the transport for this? Websockets? I have a hard time imagining 120 messages for 100 clients over websocket in a CF Durable Object, but maybe I’m just completely misimagining.

It's websockets, and yes we do make use of Durable Objects. But we do extensive batching and buffering on top of the raw platform. You can't literally send 120 messages / second to and from 100 concurrent clients. Actually the math tops out at about 8 :).

Can you explain what are the limitations in Durable Objects?

There are a variety of relevant limitations. Some of them are listed here: developers.cloudflare.com/durable-objects/platform/limits/

For this particular problem, it just comes down to CPU time. At the bottom of the stack ws.send() is a syscall and it takes significant time. You can only do so many of these calls per second. We measured it at about 8k/sec a year ago.

With a conservative limit of 2k calls to ws.send() (so that we can stay about about 50% utilization and only use half of the time for calling send), this implies 6 clients at 60 FPS using a naive approach (6 * 6 * 60 = ~2k).

It's not really a DO problem, it's just that doing n^2 messages can't really work in any platform.

This technique is pretty similar to Operational Transformations, except that it requires an authority for serialization and must re-apply any state which is out of sync.

It's not the same thing. OT works by having a discrete set of operations that are known to the system so that they can be reordered properly server-side.

Reflect is based on Server Reconciliation: https://www.gabrielgambetta.com/client-side-prediction-serve....

The critical difference is that Reflect doesn't need to know the operations up front. That is what allows developers to provide their own operations, or as we call them "mutators".

I'm using Replicache in a big project - this simple fact is a big deal. It's basically the same coding model that all the front-end frameworks have gravitated to (central store, unidirectional dataflow, mutators separate from data access) but it all stays in sync across the distributed system.

Well hello there!

Can this be used to let multiple people edit a very large image concurrently?

Depends what you mean by very large. Reflect can store up to about 50MB/room. For a PNG, that's "fairly large" I guess. We plan to increase this limit in the future.

You'd have to split up the image into chunks of about 1KB each, this would be fairly easy to do by pixels. It would be a fun project to try.

See https://hello.reflect.net/rooms#data-model for more information.

Unfortunate that's it's named after a JavaScript built-in.

Very neat project! What is the business model for this project?

I would add a link to the documentation on the homepage.

There is. Under “Get Started”

Is this much different from croquet.io?

This kills the game engine devs dreams.

Hey Aaron, cool product! Well done so far.

Curious why you think no-one has done TCR before? Also, any insights on why you're bullish on the "multiplayer web"?

Will be cool to see example docs / repos as they come out!

The examples and docs are live here: https://hello.reflect.net/

TCR is just a small generalization of what the game industry has been doing for awhile. So in that sense, we are not at all the first.

But I'm honestly not sure why it's not been done on the web yet. To me, it's a really elegant approach. It is harder to implement in a general way (like as a library) because you need to run code on the server. And as others have pointed out, it's harder to make fast.

Perhaps multiplayer is just new enough to the web that there hasn't been time for these approaches to evolve.

I mean, React was also inspired by game techniques that at that time were decades old.

What does TCR mean please?

I strongly endorse Trystero (https://github.com/dmotz/trystero) for enabling P2P communication in web apps. It’s open source and leverages public infrastructure for matchmaking.

Interesting alternative to CRDTs, "Transactional Conflict Resolution". Bookmarked.

It looks very slick, and I'd probably be interested in this if I were building a centralized local-first app. But I think this serves a fundamentally different use case vs. CRDTs. Reflect says "Your mutation code runs server-side and is authoritative" (emphasis theirs).

- If you can have an authoritative server, CRDTs come with unnecessary restrictions and overhead. Reflect presumably gains a lot of efficiency by loosening those constraints.

- Conversely, if you want clients to collaborate without a central server, you can't use a service like Reflect, since there's no server to run your conflict resolution logic.

Absolutely – CRDTs have some unique benefits, this among them. I recommended a potential user to CRDTs a few weeks ago.

But most apps people build today do in fact have a central server. And by leveraging that you can get some really nice benefits.

Yeah, it's interesting to me how from a user's point of view both technologies do something similar, but from an architecture point of view the use cases are mostly disjoint. I totally agree that something like Reflect makes more sense for most apps people build today.

Anyway, congrats on the launch! Reflect looks great. I'm really excited to see the building blocks emerge for local-first software.

If you want true concurrency, in an elegant format, with a lot of the common problems abstracted away -- use Erlang. They solved this 30 years ago.

Erlang is a programming language. Reflect is an implementation of a hosted service that helps user's build realtime collaborative interfaces.

Reflect could be built using Erlang. Erlang could not be used to replace what it does.

I don’t think Erlang solves this problem: multiplayer editing of data structures while preserving user intent.

Specifically does BEAM build in a data structure for preserving user intent while multiple users type/delete/apply styling in the same text field?

Someone should bring those logics to js and java then. Ehm, please.

What makes Erlang (and Elixir) great isn't just the actor model, it's the language-level commitment to it. Things that would mess with concurrency (say, race conditions on the reading and modification of a global variable) are simply unexpressible in BEAM languages.

Someone has!

https://github.com/untu/comedy

However to the below point, Bolt ons are never as good as baked ins. This is especially true for concurrency.

Reply on news.ycombinator.com