Hacker News Reader

Comment by szemet | Hacker News Reader

It is interesting to use fork() to snapshot application state. It seems so high-level and powerful (Compared to other C/POSIX level stuff - so I'm not comparing it to Scheme/Lisp/"your favorite high/low level environment" here!). We may collect some other examples here - just for fun:

- I know that Redis does this (RDB snaphots) too, while the child process saves the actual snapshot, the parent can operate continously relying on the system provided copy on write mechanism

- I have also found this: https://github.com/thomasballinger/rlundo - It overwrites readline library to get undo function in any REPL using it

Other examples?

View on HN

This is, incidentally, one of the built-in advantages of immutable data structures (in the languages that feature them). For example I recently had to write a test that worked with a fairly complex connection object, the execution of the test changed that object but later on I wanted to reset it to its earlier state- this was as simple as doing something like this in Elixir

    original_conn = conn
    conn = do_something_expensive(conn, params)
    assert <whatever>
    conn = original_conn # this "grabs" the earlier state, no matter how complex it is
    conn = do_something_else(conn, params)
    assert <something having to do with both calls>

Basically, if your language is immutable, then any reference to any state (no matter how complex the data structure) is the state it was at that point in time

You could say that changes to the state are forked from the earlier state.

The TeX program (by Donald Knuth) does this.

More precisely, the first version of the TeX program (now called TeX78 or TeX80) which was written in the SAIL programming language at Stanford was written in this way, because it was apparently the natural thing to do and lots of programs at the time were written like that. The present version (rewrite) of the TeX program (briefly called TeX82 but now just called TeX), was written in (a language/system based on) Pascal and meant to be portable for different OSes. So in addition to using the system's fork or dump/undump (if available) to snapshot the application state, it contains its own implementation of dumping all the relevant application state into a "format file", and loading from it. In fact, the dumping and undumping were traditionally done by different binaries:

INITEX -> VIRTEX -> TEX

See https://news.ycombinator.com/item?id=13076098 where drfuchs (David Fuchs, who was Knuth's "right-hand man" during his rewrite of TeX) describes it in more detail. (Originally posted on https://news.ycombinator.com/item?id=13073566 "The Emacs dumper dispute", and also quoted in https://news.ycombinator.com/item?id=14140421 "Improving startup time in Atom with V8 snapshots".) See also https://tex.stackexchange.com/questions/417624/installation-... for some elaboration (though it's confusing and now I realize it has at least one error).

Not exactly a fork, but Emacs is (famous/notorious) for saving its initial environment by basically coredumping and loading the coredump on startup if the startup files haven't changed:

https://lwn.net/Articles/673724/

http://emacshorrors.com/posts/unexecute.html(quite unflattering as the link suggests, but factually accurate nonetheless)

Only of historical relevance, but sure gave me a good chuckle of the "omg I can't believe that's how they made that work!"-sort when I first read it: Ultima Online fork()ed its server process(es) for marshalling in-process data into files on NFS, and used that mechanism to persist world state: https://news.ycombinator.com/item?id=9820645

Chrome and Android have both forked() warmed-up processes to speed application startup. Although looking at this 2011 post it's more complicated than I thought in Chrome:

http://neugierig.org/software/chromium/notes/2011/08/zygote....

This one is also kind of old:

http://coltf.blogspot.com/p/android-os-processes-and-zygote....

If anyone has any updates on these I'd be interested.

As far as I remember App Engine also did a similar thing to avoid reinitializing a Python VM for every request.

Web servers did this all the time to skip parsing all of your cgi scripts on each request.

https://stackoverflow.com/questions/25834333/what-exactly-is...

There is a time-travelling debugger that forks the process at each statement, allowing you to go back to a previous statement and continue from that point.

It seems like forking so often would use up a system's resources very quickly. I'd imagine that each fork needs at least a handful of individual pages for OS bookkeeping overhead, so you'd get into the gigabyte range fast even with perfect copy-on-write.

Yes you can't run like that forever. But you could break, step forward and backward and bit, and then resume. I often step forward while debugging, realise that whatever is going wrong has already gone wrong, and would like to step back a few steps.

It's more than a handful of pages - each forked process requires its own set of page tables.

Very interesting. Do you have a link where I can read more about this?

Mozilla's rr (mentioned in the article) is a time-traveling debugger, but as far as I know it doesn't use fork() for snapshots / time-travel.

rr does use fork() internally to create checkpoints, and checkpoints are created implicitly as the program executes to support time-travel. The COW behaviour of fork() is extremely helpful for performance.

There's more to checkpoints than just fork(), though; for example, when two processes are sharing memory, checkpointing the process group fork()s both processes and then must explicitly duplicate the shared memory region, otherwise the checkpoint would share memory with the original, which would be disastrous. Unfortunately this means we don't get COW behaviour for shared memory regions; unfortunately the Linux/POSIX APIs are a bit deficient here.

https://github.com/awwaiid/pry-timetravel

But I think the idea has been around forever.

gdb has checkpoint/restart that uses fork(), though you have to specify the checkpoints manually. https://sourceware.org/gdb/onlinedocs/gdb/Checkpoint_002fRes...

It has another mode where you can record and rewind execution traces, which sounds more like the description, but I'm almost certain that doesn't use fork. https://sourceware.org/gdb/onlinedocs/gdb/Process-Record-and...

It's cool but you can't un-write to disk, or un-send network packets making its applicability limited.

> you can’t un-write to disk

Why not? (Experimental) versioned file systems exist.

I think the broader point I'm making is that these time traveling debuggers can't work in the general case due to externalities.

The Factorio dedicated server has an 'experimental' fork-to-save option, which is massively helpful when late-game saves take 10-15 seconds.

Alright, I'm geared up, I'm going to go take out this hive-

> Saving

> Crash the already difficult to drive vehicle into a nearby rock

> You died

But I suppose immutable data structures can be more efficiently stored using a heap with byte-size-granularity, rather than by forking, which uses pages of perhaps thousands of bytes.

Why would you think so? A page is 4096 bytes and it's hard coded into the kernel (and the hardware). With byte sized granularity, you are just introducing extreme large overhead for page table entries and make the TLB a lot less effective.

I wasn't suggesting to make the page size smaller, but instead to not fork the compiler and use a single heap for all versions of the data.

There is another fairly similar design decision split:

LVM - low (block) level, hardware supported, generic, unavare of filesystem layer above (so it may not be optimal for some usecases) but at least this gives some separation of concerns, and still can provide some impressive features: resize, encrypt, snapshots

CoW filesystems (btrfs, zfs, ...) - specialized, carefully designed data structures/layout, can have more advanced features and more optimized to given ussecases (they have more information what is actually happening), but they are not a generic separatable layer - all the fancy advanced features are tied to the high level fs implementation

- The Resque background job queue (popular in the Ruby world) forks itself to run each job, to isolate the scheduler from any badness in the jobs.

- I’ve used fork() in Ruby for similar reasons when a library had instability or memory leaks and I wanted to isolate it rather than try to find the bug (I’m looking at you, ImageMagick).

A while back, MRI Ruby got an improved garbage collector that keeps the mark bits separate from the objects just so copy-on-write would (mostly) work in cases like this.

Reply on news.ycombinator.com