Hacker News Reader

Comment by Something1234 | Hacker News Reader

Can you name a use case for a fast non guaranteed write?

Analytics: you don't want to slow down your app, and you don't care if you lose a few records in the process.

Importantly, in analytics workloads, it is very important to know roughly how many writes aren't making it. Otherwise your analytics system sucks.

Interesting. How would one know that?

Good question. You'd need some accurate-enough data source telling you about failed writes. Which eventually comes back around to needing a consistent database and indications of client disconnects.

With a huge amount of data (as I've heard analytics is), could you take a sampling approach where you log every n transaction and only check those against the DB?

Caches. If you lose a write, you just get a cache miss.

Periodic snapshots of state held elsewhere. If you lose a write, you just get stale data until the next update.

Firm realtime work. If you lose a write, that sucks, but a slow write sucks just as much.

Sure. Data that people don't care about enough to be worried about losing--for example, time series data from an unimportant remote sensor. Should this data be recorded at all? Maybe not, but if should then a best-effort recording may be fine. It may even be all that's possible.

I wouldn’t go as far as to say an “unimportant” remote sensor... but I think you’re correct in spirit.

I could think of an instance where you’d like to log data, but the occasional datapoint being missing wouldn’t be terrible. Maybe something like a temperature monitor — you’d like to have a record of the temperature by the minute, but if a few records dropped out, you’d be able to guess the missing values from context. Something like the data monitoring equivalent of UDP vs TCP.

Even more elementary that sibling comments, this also happens in gaming all the time. You are recording live results, say in Fifa, but if you unplug your device, your results are gone, since they were memory only. The game simply cannot afford to write to disk, the write is "non guaranteed" in the true sense of the word, but it is fast.

You then "checkpoint" when the game is over.

You might dissent that is not a "non-guaranted" write, because in fact the write did occur, but I simply want to allude to the concept of a "non-secured" write, in that it vanished without an fsync.

The number of likes in a given post in your favorite $social-network-of-the-year.

Since that number may well already be approximate if it's recorded in a hyperlog log.

Telemetry.

I work for a telco where we log large amounts of network requests using MongoDB.

Reply on news.ycombinator.com