Hacker News Reader

Comment by noelwelsh | Hacker News Reader

Holy return of tuple spaces! Are there any features for controlling consistency, or is it just YOLO last write wins?

I had to upvote you just because I plan on getting some mileage out the phrase "YOLO last write wins." It has just the right amount of snark to remind junior developers to not trivialize or discount the value of consistency.

And it's really "YOLO last write recorded wins," because who knows what order anything arrives in.

"yolo a latter write probably wins ... I guess?"

Seriously, if you're new to that domain go read:

https://aphyr.com/posts/333-serializability-linearizability-...

https://aphyr.com/posts/313-strong-consistency-models

Last write wins. There are several features for controlling consistency that we have been prototyping. Based on how storage is used we will enable those as the product matures.

So why not strong eventual consistency and CRDTs? As they fit perfectly for such functions running on edge nodes and no silly limitations of 1 write per second necessary.

Indeed. This is just stage 1 of our storage plans. We have some really cool stuff in the works but we wanted to get basic KV functionality out there for people to start using ASAP.

OT: we need to read more about your website-in-kv-thingy setup.

It was pretty trivial. I just stored each page into a value and served them out of there, with some stupid code to infer content-type from file extension. I meant it as a dumb demo because I wasn't feeling creative. I didn't know John would mention it in his blog. :)

Currently this isn't a great way to serve content because the values are limited to 64k, you have to infer content-type (which makes your URLs ugly), etc. So I don't recommend it. We'll come up with something better in the future. :)

Yes, that is one of the models we have been considering.

So, with a global consistency window of 10s, and 1 write per second, would consistency ever be reached if a key is written to every second? What value would be visible?

One great example of why this would matter to me - could workers ever institute a coordinated rate limiting function using this KV store?

One thing to remember is you have at least two other ways of storing values. Each worker has global memory which persists between invocations (local, atomic), and you have access to the Cache API (per-data center, ephemeral).

By combining those with KV to do global coordination on a more course time scale, I think you could.

Rate limiting doesn't have to be coordinated to work. But you do need a PN-Counter, which they are only considering at this point.

A PN-Counter is a method of coordination. And if the writes were coordinated with the consistency time, that would be sufficient to implement consistent counters (or even a naive counter in a store given atomic operatons).

PN-Counter is coordination-free counter, it can't be used as a method of coordination. But it's ok for global rate limiting, as it doesn't have to guarantee exact rate at any point in time, as long as it eventually averages on specified rate.

For anyone curious about this, it probably all comes down to an atomic compare and swap request:

https://en.wikipedia.org/wiki/Compare-and-swap

It might be straightforward to do this with an ETag like Firebase:

https://firebase.googleblog.com/2017/07/introducing-conditio...

Not sure what this would require on their backend though.

Yes, you'll be able to use etags with if-match for optimistic concurrency. When reading and writing from a Worker, if you do a get() followed by a put() on the same key in the context of the same request, we'll take care of the if-match automatically under the hood, so the put() fails if there was a concurrent write.

(This logic is not quite in production yet but should be within a week or two.)

Can you implement IPC with this system? It seems wrong to relate this to tuple spaces otherwise.

A poor man's IPC, like HTTP and HTML is a poor man's application platform ...

Put the target channel name and a message id at the front of each key and you can use a tuple space for IPC.

What I mean is: can a process block until a particular key appears, then resume execution after receiving the associated value? That's the main differentiating feature of a tuple space, and it wasn't clear to me whether this system supports that or not (or even has a concept of a process).

No, that's not supported at present, but that's an interesting idea.

Workers don't have the concept of processes. The unit of compute is an individual event -- e.g. handling an individual HTTP request. So we'd have to augment it a bit, but we can probably come up with something useful.

Reply on news.ycombinator.com