Comment by gravypod | Hacker News Reader

Comment by gravypod | Hacker News Reader

I wish we took the path of EROS [0] rather then "RAM and DISK are seperate". A lot of problems stem from that incompatable viewpoint of computing. Computer Science is about hiding complexity under lays of abstraction that continualy provide safer states and constraints on the things built on top of them. Our abstraction that RAM and DISK are seperate is not safer nor does it provide constraints that are simple to navigate. Thinking about this the other way, where DISK is all you need and memory is just a write-through cache, is much safer in my opinion and leads to some really cool application design.

If RAM and DISK are the same, then writing a file system is just writing an in-memory tree. No need to pull data from the disk, just navigate the tree in your program's memory and pull the blob data out. Want to persist acorss reboots, protect against power outages, or save user settings? Just set a variable and it'll be there.

The benifits are much better then the costs.

[0] - https://web.archive.org/web/20031029002231/http://www.eros-o...

The AS/400 (or whatever they call it now) had an approach like that. Everything was on disk and RAM was just a cache of disk. That also meant every "object" had an address and could be accessed by any process with suitable permissions. There are lots of other things they do, with a very different approach than Unix, Windows etc.

Frank Soltis' book is recommended reading: https://www.amazon.com/dp/1882419669/

AS/400 is really an amazing system, in many ways still ahead of its time. Persistent, single-level storage and capability security are ideas that still have yet to catch on in the mainstream—even though more research gets poured into NVRAM every year.

It's a shame hardly anyone knows about it. Those things are a joy to use. You can get a free (limited, but still useful) AS/400 user account to play around with at http://pub400.com/. I really recommend it.

(Disclaimer: I'm slowly working on a system that resembles AS/400 in many ways, but optimized for analyzing and reporting on very large timeseries databases. It's intended for business applications that require a combination of scheduled reports and fast ad hoc analysis of big timeseries data, initially the oil & gas industry (which is where I work in my “real job”).)

The challenge with this is that abstracting away disk in a way that isn't horribly leaky is incredibly hard as long as one lets us manipulate individual bits and the other requires us to write whole sectors.

Note that EROS is not providing a write-through cache. It's providing a write-back cache using checkpointing coupled with a journalling capability and ability to explicitly sync data.

So it's leaky: Your application needs to know that it needs to structure it's writes to memory so that they will make sense if the system comes back up with some of the data missing, and needs to know how to use the journalling functionality.

It can't just act as if it's running in RAM forever.

I don't know where you get the idea that you can't just pretend you're running in RAM forever. If you look at the main goal of EROS you can see that is the point.

Check out http://wiki.c2.com/?TransparentPersistence

From the EROS website which specifically described the checkpointing mechanism as well as gives a short description of the journalling support.

If you just pretend you're running in RAM, and the system crashes, you will lose the data between the crash point and the last checkpoint unless you have explicitly used the journalling mechanism. Often that is acceptable. E.g. since you're restoring the program at the same point in time, if the changes are entirely based on data that were in the system at the point of the last checkpoint, it will just redo the work to calculate the changes.

But if there are side-effects, that is often not going to be acceptable. E.g. database updates that the system has said were committed will suddenly disappear.

To solve that, EROS has a journalling mechanism to allow you to give guarantees about specific data that changes in between checkpoints, but that requires applications to explicitly use it to tell the OS what needs to be saved when so that the application can guarantee that a given piece of data has been durably recorded when it promises a client it has been recorded, and that the writes get correctly ordered.

That's a sensible compromise - if you do it right, it only needs to touch the "boundaries" where the system does IO.

> If you just pretend you're running in RAM, and the system crashes, you will lose the data between the crash point and the last checkpoint unless you have explicitly used the journalling mechanism

> But if there are side-effects, that is often not going to be acceptable. E.g. database updates that the system has said were committed will suddenly disappear.

Yes, not every system will forever be recoverable. You can definetly crash at just the right moment to ruin your year. I'd still like the other safety constraints that this provides because I still think that even if N (where N is the number of threads available on the CPU) processes have a chance of being corrupted we're still going to be saving the rest of the processes on the system that aren't currently transacting with one another.

I absolutely like many of the ideas behind EROS, including checkpointing, though I think it does have some issues. E.g. we often treat reboots as a "clear all state to recover from weird situations", and so we still need something like that.

The point is more that it doesn't mean you can just treat things like we do RAM now. It ends up being closer to how you'd work in apps that use mmap'd files to back persistent data.

The capabilities model was interesting too.

I think my biggest problem with a persistence model like that, though, is that I suspect it would encourage not thinking seriously about state, and ideally I'd prefer a system where state is minimized. E.g. compare EROS with the virtual opposite: Android, for example, where apps might find themselves killed anytime. Some apps handle it poorly and go through lengthy initialisation processes again when restarted, but many maintain the illusion of being fully presistent to the extent that users can rarely tell if they've started from scratch or not.

I'd love to see more work into allowing the illusion of a persistent process with little developer effort, though. Perhaps OS-level per application checkpointing support that the application can have control over (allowing the app to control exactly what gets checkpointed, and when to ditch the checkpointed state to reinitialise). So "cleanup" can occur by restarting processes transparently for users while hopefully providing most of the benefits of persisting state.

Perhaps coupled with OS-level checkpointing of the information required to bring said "virtually-persistent" apps back up in the same state after a reboot.

You might want to investigate Mumps:

https://en.wikipedia.org/wiki/MUMPS

Setting data in memory is the same as setting data on disk, the only difference is the name of the variable:

s X=1 ; store 1 in variable named X, in memory.

s ^X=X ; store 1 in variable named X, on disk.

s X=^X ; load disk to memory

How is this different than just memory mapped files? I guess it happens a little more automatically, but it doesn't seem to really solve a major problem that I can see.

Have you ever lost power and lost data from a document you were editing? Has a server ever crashed in a datacenter, it's data corrupted, and now your company has lost a few 100k to a ffew million? Have you ever had to wait for processes to start again for a long time after fixing the hardware failure?

These are all problems that have been solved on EROS based system. They used to do demos where they would setup a system and have someone start working on some code or a text document, they'd pull out the power plug of the system, plug it back in and the user would be right were they left off. No data loss, no corruption, just back to work.

None of that was handled in user space. That was all opaque and you didn't have to worry about it at all.

How is that not slow as balls when you're hammering memory? Guaranteeing atomic writes to disk for every memory access would seem to be problematic from a performance perspective.

How is paging not slow as balls? When you're done changing your data, or your time quantum is up, you are paged and saved. The only difference now is if the system dies for some reason, you come back right where you were paged off.

So it's not right where you left off, and the program state is unknown?

I'm guessing the system must effectively "checkpoint" your work regularly and sync to disk to avoid partially saving a state and corrupting the data.

This isn't terribly different from working on a memory mapped file except that it also saves the ephemeral state of the running program so it can be restored. But I still don't understand how it's not going to be horribly slow when you start your program and the first thing it does is allocate 4GB of memory for its workspace. Synching all of that data to disk is a massive undertaking, and this isn't an uncommon use case, people start virtual machines all of the time.

And paging is slow as balls. That's what this whole article is about, modern machines are unusable when they start paging.

I've read somewhere that for BeOS demos they used to play a bunch of videos and music and then unplug/plug and after boot everything was playing again from where it left. I guess they were using the same design for process persistence.

If "never loose data" isn't a great selling point then I don't really know what is.

It's a trivial problem if you're willing to run your system entirely off of the disk. I mean the performance will be unbearably slow, but you'll never lose your data.

That was just the media player remembering where it left off and restarting from that place. There may have been some metadata support in Be's filesystem to help that, but it's not technically necessary. It's about as amazing as your web browser reloading your tabs when you restart it.

Memory mapped files are incredibly hard to use for consistent, durable storage. I mean, so is POSIX I/O in general, but if you do MAP_SHARED you made your life even more complicated. (MAP_PRIVATE and rewrite-the-whole-thing-for-every-commit works, though, and can have some advantages).

Reply on news.ycombinator.com