Plane: Per-user backends for web apps
driftingin.space
driftingin.space
Plane came from our desire to build tools that have the low friction to use of running in the browser, but use more memory and compute than the browser will allocate. The basic idea is to run a remote background process, connect to it over WebSocket, and stream data.
This ends up being a surprisingly useful primitive to have, and it's been used to:
- Pixel-stream X11 applications to the browser over WebRTC[1]
- Run IDEs and notebooks
- Power Figma-style realtime collaboration backends (including https://rayon.design).
Here's a direct link to our repo: https://github.com/drifting-in-space/plane and docs: https://plane.dev/
[1] https://twitter.com/drifting_corp/status/1552773567649091584
The sooner you can figure out Windows, the better.
Because…
There are many ways to implement multi-tenant SaaS; but a highly underrated approach is to write a single tenant app, then use infrastructure to run an instance of it per (currently logged in) customer. Plus a persistent database per customer of course.
This has the tremendous advantage that you can add another column to your pricing page easily: the "bring a truckload of money and we will set you up to run this behind your firewall" tier. There are still a lot of orgs out there, large ones with considerable financial capacity, who really want this.
Often times an overlooked aspect of this is that, even without a freemium option, that revenue can be 0. Consider all the demos for potential customers, and all the examples setup for testing, etc. these will all cost money, and if they can’t be shared, than those costs may make it unworkable on a per-instance basis.
The idea is to only allocate a running instance while there is at least one active user of that instance. So for occasional-use apps the ongoing cost per idle customer would be just the storage cost of the database, very close to 0. Obviously for something like a chat app, email app, anything else that people tend to leave open all day, a different approach is better.
That first user is going to have a bit of a wait while you turn on their server. Maybe you keep some empties warmed up that just need a reconfig and restart so it’s fast.
Personally, I’d only do any type of multi-tenant if the cost of some micro instance behind an auto scaler was negligible to the value of the contract anyway.
I think that would work fine for a lot of other apps, at least where you're looking to start a lengthier session.
It's all on 1 server, just a Go https proxy launching firecracker on demand, no k8s.
If I were optimising, I'd first make sure the root filesystem copy ops happed on a CoW filesystem, then change the readiness polling for a "long" poll that's under the server's control. And next buy a faster server :) But I'm using a relatively heavyweight userspace app, so I'm curious if you can see there might be other gains to be had.
Our application is using VS Code as an environment for programming exercises. So even 30s would have been fine when people are using it after that for 1-2 hours.
If I could get it down to 2s I'd certainly enjoy testing it more!
However many environments you have of Dev, Staging, QA, Integration, whatever is the same as normal. Sales gets an instance, but I’ve seen that anyway and it’s just one more. Can have a public Demo. As long as you don’t scale O(n) with prospects, it’s not a meaningful cost. Even if you did that, I’d bet it’s a tiny fraction of your cost of sales.
I don't have much to say about the 2nd one, but my team does the first one for our (internal) customers, so we deploy and manage in our own accounts a whole service stack for each of our customers. It's a nightmare.
We are going to do some rearchitecture work next year to move away from it, because it drains so much of our time in operational load.
One example of a major pain point that we have is encountering random failures in 3rd party services, such as AWS. For a more classic service that you deploy 2 or 3 times a week, if CloudFormation deployments fail once every thousand deployments for random errors, you'll have one failure per year. Well, we deploy thousands of instances of our service 2-3 times a week. So we have failures every single time we deploy.
Oncall is a nightmare (I don't love oncall anymore, I created my login in my previous team) because we just fix a tsunami of similar tickets that are each from a different instance of our service, for a single customer, but often with a different root cause.
We probably have half of our headcount dedicated to initiatives that wouldn't need to exist if we had a more classic 1 service = 1 instance approach.
Just don't do it.
Whether on site installs are a good thing to offer, of course depends on how valuable that is to your customers versus the cost/effort to support it.
As you point out there are major trade-offs! If your product architecture requires substantial multi-service complexity per tenant, that points away from the instance per customer strategy.
My work deploys an instance per customer, and I haven’t encountered the same problem, but they number in the tens at this stage so not at the same scale yet.
Would love to hear any dos or don’ts you’ve picked up, being further ahead in terms of scale!
To be honest it comes down a lot to the tools you are using, you have to make sure they scale. For example we deploy one CloudFormation stack per customer (3-4 actually but it's not important), and those are deployed all as part of a single deployment pipeline (think AWS CodePipeline type of thing).
Well, turns out this pipeline thing does not like deploying to thousands of targets at once and only supports a few hundreds at most. We had to refactor the way we do that part because just loading the page with all the targets was taking 5 minutes. We also had to split our targets across 2 pipelines, and soon we'll have to go to 3, and it's a pain in the ass and a big engineering headache every time.
The main issue comes from operations, invest in automation ASAP, and make sure you make your customers accountable for their own mistakes. Half of our tickets are raised because a customer changed something on their side without thinking of changing the configuration in our system, which then breaks. If we alarmed them instead of ourselves it would be a lot easier.
I think "per-user" is probably the wrong killer feature for something like this. Much more potential in shared distributed processes that support multiple users (chat, CRDT/coauthoring). Appears that the underlying layer can probably do that.
In any case, super cool idea, and I hope something like this lands in the serverless platforms from all the major cloud providers. It's always been mind blowing to me that Google Cloud Functions supports websockets without allowing you to route multiple incoming connections from different users to the same process. That simple change would unlock so many useful scenarios.
We actually initially built this on Kubernetes, twice. The MVP was Kubernetes + nginx where we created pods through the API and used the built-in DNS resolver. The post-MVP attempt fully embraced k8s, with our own CRD and operator pattern. It still exists in another branch of the repo[1].
Our decision to move off came because we realized we cared about a different set of things than Kubernetes did. For example, cold start time generally doesn’t matter that much to a stateless server architecture (k8s’ typical use), but is vital for us because a user is actively waiting on each cold start. Moving away from k8s let us own the scheduling process, which helped us reduce cold start times significantly. There are other things we gain from it, some of which I’ve talked about in this comment tree[2]. I will say, it seemed like a crazy decision when I proposed it, but I have no regrets about it.
The point of sqlite was to allow the “drone” version to be updated in place without killing running backends. It also allows (but does not require) the components of the drone to run as separate containers. I originally wanted to use LMDB, but landed on sqlite. It’s a pretty lightweight dependency, it provides another point of introspection for a running system (the sqlite cli), and it’s not something people otherwise have to interact with. I wrote up my thought process for it at the time in this design doc[3].
You’re right about shared backends among multiple users being supported by Plane. I use per-user to convey that we treat container creation as so cheap and ephemeral you could give one to every user, but users can certainly share one and we’ve done that for exactly the data sync use case you describe.
[1] https://github.com/drifting-in-space/plane/tree/original-kub...
[2] https://news.ycombinator.com/item?id=32305234
[3] https://docs.google.com/document/d/1CSoF5Fgge_t1vY0rKQX--dWu...
Also you should give some advice about how to deploy when the default for deploying apps in an organization is K8S, what might be not too exotic nowadays. Will Plane need it´s own cluster? Does it run on top of K8S? How is the relation to K8S in general for a deployment scenario?
THANKS!
Re. the “Why not k8s” question, you might enjoy this post from a couple months back; although it only touches on Plane briefly, it shows the framework we used to make the decision. https://driftingin.space/posts/complexity-kubernetes
I also don't read the GP comment as being negative toward SQLite. It sounds more like the author was surprised about the architecture, since a naive view would think Kubernetes would be good enough.
Knative has solved most of those pod start time problems since it’s dealing with a similar scenario, unless 0.008s startup time isn’t good enough for you.
I think you would get a much higher long term payoff with a custom scheduler. Dask does something like this. Both on scheduling and when it has to "Drain".
I built a similar service for AWS back around 2015 to do web-based pixel streaming applications. That service's legacy still lives on today in many descendants, but I was always bummed that no team was willing to invest in making it generic. Everyone who needed it either forked it or re-implemented around the original design.
Warms my heart to see something like it on the outside. It's a super powerful concept. Great work!
My impression of that approach is that it’s good for IO-bound work and stateful business logic, but less so for the CPU/memory-bound applications that we’re targeting. I’d love to know if there are counterexamples to that though. It’s admittedly been over a decade since I touched Erlang and I’m not up to date, only peripherally familiar with Elixir and Phoenix.
* run a full blown browser in the cloud and stream back pixels
* emulate a native application in the browser (like, github.dev)
Both are okay but per user backends feel like a nicer primitive to build applications with. Apps that run locally in the browser but can access cloud compute/storage on demand.
v2.0: per-user backends for web apps hosted on edge compute.
v3.0: per-user backends “hosted” as desktop software (the ultimate edge cloud)
If the user does nothing the process still ticks along. It may be doing background work or doing nothing. It can keep state and periodically save. It is like a desktop app experience on the web, if you like.
If each user or org is doing their own thing and the service is not too “social” requiring cross interactions (so more like say a CRM than a LinkedIn) I think it is an interesting model.
Let alone slow feature roll outs!
An example case might be to write in a notes app locally, and collaborate on it, and then have it saved and backed up to re-use on different machines. I'm guessing that'd probably be a layer on top of this, or outside of it.
Can this be used, for example, to create a collaborative whiteboard app? I am trying to understand the use cases. Does it depend on Docker and Guacamole? Guacamole was mentioned in passing in your video but I haven’t heard of it before.
Yes, it could be used to create a whiteboard app by spinning up one backend per whiteboard, and connecting each user to the backend of the whiteboard they are viewing. For data synchronization, where it really shines is in slightly more complex cases where data updates need to be validated (e.g. 3d software or a CAD app), because unlike a traditional pub/sub service which just passes messages through, you can run your own code on the backend.
It depends on Docker, but Guacamole is only used for the demo. Guacamole is what lets us run Firefox inside Docker and then view it inside another browser so that we can test an install without having to configure a bunch of DNS and certificate stuff.
Of course, one of the big differences with initd is that it runs on a cluster of machines instead of locally, which turns out to be most of the difficulty.
In addition to the a11y benefits, this fits our general vision of how apps should work, which is that you shouldn’t need a round-trip to the edge to do a UI paint.
that said, it's nice to abstract it away, and with a container - makes it very generic which is nice.
will definitely look into it for a project.
btw how decentralized can the servers be. potentially can anyone join in as a server. discovering peers will be another interesting challenge.
Does the service expose time series metrics?
How would I detect and remedy a hot shard?
Are resource caps well defined before hand/are all instances expected to have similar resource consumption?
What would administratively draining an instance look like?
In general, Plane backends are meant to be used with thick clients, so there’s also the option to treat clients as nodes in a distributed system for the purpose of failure. If the server goes down and is replaced, when it comes back up, the nodes could buffer and replay any messages that may have been lost during the failure. Over time as we see patterns emerge, we may create frameworks from them (like aper.dev) to abstract the burden away from the application layer.
Time series metrics are exposed through Docker’s API, collectors for it already exist for various sinks. We will soon be sending some time series metrics over NATS to use internally for scheduling, but the Docker API will be better for external consumption because the collector ecosystem is already robust.
Resource caps can be defined at “spawn time”. They are not expected to have similar consumption, but the scheduler is not yet very smart, our approach currently is admittedly to overprovision. The scheduler is a big Q4 priority for us.
Draining currently involves terminating the “agent” process on the node, which stops the drone from advertising itself to the controller. Traffic still gets routed to backends running on that drone. We have an open issue[1] to implement a message to do this automatically.
For what purpose?
> Re. handling failure
There are several operations that should be near seamless and very well thought out/handled reasonably including for the pieces of Plane itself.
Push new code
Roll back code
Push Canary code
shutdown -r now
add machine
remove machine
cluster wide restart
replace master
add replica
backup/restore backup
A single point of failure is an extremely convenient architecture, but it is also a brittle and pain causing architecture that will resist scaling and the clean-ness of operations is probably the best window to assess that.
As far as architecture itself goes, why choose to use DNS rather than header based information/cookies? Why let the client choose the backend rather than hiding that as an infrastructure side implementation detail?
I'm curious if this can be used to stream an entire X11 Linux desktop to the browser?
(I work at drifting in space)