Introducing Workers KV

So why not strong eventual consistency and CRDTs? As they fit perfectly for such functions running on edge nodes and no silly limitations of 1 write per second necessary.

So, with a global consistency window of 10s, and 1 write per second, would consistency ever be reached if a key is written to every second? What value would be visible?

One great example of why this would matter to me - could workers ever institute a coordinated rate limiting function using this KV store?

And it's really "YOLO last write recorded wins," because who knows what order anything arrives in.

Put the target channel name and a message id at the front of each key and you can use a tuple space for IPC.

A poor man's IPC, like HTTP and HTML is a poor man's application platform ...

Yes, you'll be able to use etags with if-match for optimistic concurrency. When reading and writing from a Worker, if you do a get() followed by a put() on the same key in the context of the same request, we'll take care of the if-match automatically under the hood, so the put() fails if there was a concurrent write.

(This logic is not quite in production yet but should be within a week or two.)

I could be understanding the crux of your question wrong but I believe the point the article and technology makes over and over is that it offloads nearly all of the stress and communication off of the backend and instead replaces it with a Worker KV which converts into becoming one static call and then everything is handled within the Worker. The applications seemed vast and only limited by creativity, as well.

One write per second, per key plus eventually consistent does limit the functionality a bit.

You're right for some use cases though.

You work for Cloudflare so that's a fun question but realistically there are still plenty of limitations. The script size, number of routes, KV consistency, write limits, and simple get/put API, etc.

It can work for simple apps but there's a long way to go before any serious enterprise system will be hosted entirely on a FaaS system like Workers.

> why do you need an origin at all?

At the point that CF replaces the origin, it is the origin.

The confusion in your query is the idea that "origin" is a rigid, fixed concept rather than flexible.

Reduncy, avoid vendor lockdown, Worker KV is amazing. On the billion scale it is expensive.

Accidentally serverless.

The simplest example is liveblogging - someone is sitting at the next iPhone launch, and sending out photos and commentary that's being updated as they send on a large number of readers' screens worldwide. The Verge has a good system that they do with polling S3 for new data, but there's really much more effective ways to do it.

The more specific cases include opening ticket sales for movies in India - for some big movies tickets are sold out really quickly, so there'll be hundreds or thousands of people with browsers / phones open on a seat layout and they all need to see which seats are being sold in real-time.

Then there's broadcasting stock market movements and tips to thousands of traders in real-time.

The only existing solutions that seem really scalable are fanout.io - others like pusher.com and pubnub.com have pricing that's not conducive to serving a large number of simultaneous users / broadcasting. Fanout.io gets this right, but would be nice to have cheaper alternatives / make it a commodity.

Given that the CF nodes do support websockets, I assume they terminate and re-create new connections to origins as well - so the nodes are capable of holding a large number of websockets open, right? Can Workers intercept websockets / stream data into them? And given that the KV store is capable of propagating new values to all the nodes, there could be a way expose incoming change notifications on keys, which would handle this use case. Could also implement as continuous addition of new timestamps keys, prefixed with a topic.

Best link is probably the blog: http://blog.plan28.org/

Awesome, I'll be sure to keep up on that.

Thank you! Lots of super cool stuff coming out of CF these days, the pace is hard to keep up! Not that it's a bad thing necessarily :)

At present, yes, lazy propagation. But we're also exploring ways to do proactive propagation where desirable.

Thanks, I misunderstood that part... very cool indeed. What are the overhead limits for workers?

Aside, it seems like this could be really interesting combined to make an entire API with static delivery from S3 or Azure Blobs, with a backend on a cloud hosted database. With a lot of flexibility in between.

The shopping cart is still stored somewhere, but when the user has cleared their cookies, etc., what information do you have that will let you it's their shopping cart so you can find it again?

There's a (K,V) pair somewhere, but in order to get V, you need K, which you've lost.

It's not finding a parser that I find problematic, it's the dataset needed. My understanding is that the workers are limited to 128MB of ram and that they may be 'restarted' at any time. Datasets for accurate browser detection would be really tight in fitting into that amount of RAM when combined with whatever else you may be doing and you have to deal with load / parse time whenever a worker is restarted. This seems like a common enough need that cloudflare could provide it as an API and their implementation wouldn't have such issues.

What you're describing would work...

well, for one example off the top of my head would be using geohashes for proximity in records... if you could glob every geohash with the same first 6-8 characters, you could do range queries pretty quickly.

I've considered similar for shard keys in terms of proximity related information.

Heck, I think that's well within hobbyist budgets. Thanks for pricing it out nicely :).