Hacker News Reader

Comment by badsectoracula | Hacker News Reader

> Next, his problem was that the database would get corrupt sometimes when something bad happened in the middle of writing the file.

I'm not sure i understand how this can happen... unless you try to update JSON in-place (which is a very bad idea for any text-based format), what you do is encode/write the entire JSON from scratch. So either the file is written properly or it isn't written.

Honestly from the entire message it doesn't sound like JSON was a bad idea but that your coworker didn't know what he was doing and if he was doing something else then he'd still be doing big mistakes.

View on HN

Here, have a read. Warning: if you haven’t done low-level development, you might walk away wondering if your entire life has been a lie.

https://danluu.com/file-consistency/

Truly amazing read. Thanks for sharing.

I learned some things! Thank you. :)

I've read that at the past but this article has an issue:

> if there's a crash during the write

It never brings up how can there be a crash in the first place (also the entire article is too filesystem specific).

Any crash. Power failure, hardware failure, kernel panic while working for another process, software bugs, etc.

It's impossible to make crash-free systems. If your goal is to maximize the chance that your data remains valid, you have to plan on that.

1. A cosmic ray storm turned all ASCII charactees into ECBDIC

2. Lightning struck the 12 V feed and upped the voltage to 10 MV, turning all 0 and 1’s into 6’s

3. Someone spilled a New England Pale Ale on the server

4. The process was assinated by the mysterious killer only known from his modus operandi of leaving OOM written in blood across the syslog

5. Birds nested within the server and fed all the SATA cables to their babies

Seriously though, disk writes aren’t atomic.

File renames are atomic. This is a solved problem:

1. Write your updates to a copy of the file.

2. Do an atomic rename of that copy to the original.

Renames aren't atomic on crash.

https://danluu.com/file-consistency/

They are if you perform the correct sequence of fsync operations on both the file and the directories, and use a file system which is correctly implemented.

Renames aren’t atomic on crash.

What crash?

The article linked above never explains that part, it only assumes that it will happen. From the code it sounds as if the crash can happen in the OS itself (but then the entire kernel will crash). At that point things are completely outside your control and you might as well running on broken hardware.

Hmm...unlike the rest of the post, he just asserts this without support.

His sources seem to disagree, certainly with that kind of blanket statement. For example:

Our study takes a pessimistic view of file-system behavior; for example, we even consider the case where renames are not atomic on a system crash.

So this is clearly considered an outlier/unusual.

(e.g., a single 512-byte write or file rename operation are guaranteed to be atomic by many current file systems when running on a hard-disk drive)

[https://www.usenix.org/system/files/conference/osdi14/osdi14...]

I remember reading quite a bit about the (performance reducing) lengths filesystems go to in order to ensure consistency of directory entries even in case of a crash, and for example how "soft updates" were introduced to accomplish the same consistency with less of a performance degradation.

Looking at it from another angle, if you are running on top of a filesystem that cannot keep itself consistent, then you are SOL, there really isn't anything you can do to mitigate.

Just like we can't guarantee that we will be able to persist data that's in memory to disk if the OS is free to kill us at any time. "Best effort" it is, which means getting the data to disk as quickly as possible and not corrupting what is there.

1. Write your updates to a copy of the file.

1a. fsync() the file to ensure the contents are durable.

2. Do an atomic rename of that copy to the original.

2a. fsync() the directory to ensure the rename is durable.

Or just use a database like SQLite or Postgres that has developers dedicated to solving this problem.

Disk writes aren't atomic but you know if they failed or not (unless your OS is lying to you but that is a problem with the OS, not your code).

Reply on news.ycombinator.com