Hacker News Reader

Comment by cliveowen | Hacker News Reader

I so much want this to be true, I can't wait for it. Not having a layered memory architecture means no VM, everything has be rethought from the filesystem down. On a related note: "Operating systems have not been taught what to do with all of this memory, and HP will have to get very creative." Anyone knows what this could possibly mean?

View on HN

Well, you wouldn't have to save files. You could just use a data structure in main memory because it's non-volatile. But how would you share data between programs when there's no disk? And you could execute binaries directly from storage without "loading" them into memory first. But you'd have to keep an original copy somewhere in case it modifies itself as it runs.

You'd probably still have memory partitioned into persistent areas and runtime heap of a sort, and you might very well continue to use the file system metaphor because it still makes sense to humans. Whether a file is bits on magnetic disk, in flash memory, or fast persistent RAM isn't all that meaningful to the user. As for sharing, I don't see why this would be an issue. If you have to move memory contents between machines then you still need some sort of external transport, whether it is a network, or a glowing glass cube.

Took the words right out of my mouth (better said, of course).

Filesystem semantics are a bottleneck. Fusion IO had to get rid of them when they demoed their billion-IOPS platform last year. https://news.ycombinator.com/item?id=3434711

For communicating data between programs, would you create a filesystem object that points to memory inside your program, or what?

Yes. On Linux it's /dev/shm and it is a ramdisk that you can access from multiple processes.

IPC (Inter-Process Communication) is a known technology. This won't change it much if at all.

That doesn't make any sense. Create a file, memory-map it, bam, you have RAM performance with "filesystem semantics". (There is NO overhead associated with memory-mapping a file on a ramdisk in the Linux kernel.)

Probably what they're referring to is something more along the lines of replacing SCSI or NFS with a protocol that is a better match for the random access patterns used to access in-memory structures. (Hence their 64-byte I/O size; that is equal to a cache line on Intel architectures.)

Hm, how would this memristor stuff connect to the system? Will it look like a memory controller or more like PCI-e flash?

It sounds like HP is pitching it as memory.

Continuing to use the file-system metaphor (Instead of, I don't know, anything else, perhaps some kind of global namespace with built-in versioning like Urbit's) seems like lobotomising this incredible new paradigm before it is even born.

It might be. I'm not arguing that it is the ultimate best metaphor to use, but these things do tend to be adopted organically and become very entrenched.

Some kind of git-like diff-based snapshot system for programs?

I mean, there should still be a need for both the data and code sections of a program. Similarly, some kind of tree-like structure for naming "files" locations (for interprocess communication). I would imagine, at least in the early days of some diskless architecture, we'd have something that looks very similar to how computing is currently done but in one memory space.

What's the issue with RAM disk to address your concern about "no disk"?

It's a lot of unnecessary copying. The data is already in RAM, but you "save" it to a different RAM location, in the RAM disk. Or if you do all your work directly on the file in the RAM disk, you have to deal with filesystem overhead constantly.

Or if you do all your work directly on the file in the RAM disk, you have to deal with filesystem overhead constantly.

Not so. Take a look at the Linux kernel implementation of memory-mapped ramdisk files. There is no "filesystem overhead". Zero. The kernel pretty much sets up the page mapping and then is hands-off.

Even if your memory mapping is backed by a file on a block device, the only overhead is occasional paging to disk. This happens on the order of SECONDS… not "constantly"!

That's assuming you already have a file and know how big it should be. Or does it take the whole space for one file, and every program on the computer shares the space?

Files can be both sparsely and dynamically allocated.

You save things to persistent storage because you want the copy. In the large majority of cases the format on the filesystem is different from the format in memory because the file format is meant to be more readable by humans or by machines with a different endianness or pointer size etc., whereas data in memory is binary and platform-specific to optimize for fast access by the local machine. The benefit of that distinction doesn't go away just because you make the persistent storage faster.

I think there would still be a need for "files", ie, a way to store a self-contained document format in memory that is independent from the code that is working on the data at one exact moment: to allow moving documents between computers, but also to allow updating the software in a safe way (ie without losing data), to save older version of a document, but most importantly IMHO to be able to create snapshots that can be safely reloaded in case of a system crash. We're not ready to have systems that can't be rebooted.

Filesystems aren't going to disappear. Databases haven't disappeared, even though there exist 256 GB RAM monsters in data warehouses.

The logical structure of a filesystem is still a useful abstraction (and has been decoupled from physical constraints for a long time now).

And you could execute binaries directly from storage without "loading" them into memory first.

Linux already does this. Program code is cached copy-on-write in shared memory. The fact that, say, bash has to be loaded from disk once is pretty much irrelevant. After that, the kernel's just managing pointers.

Virtual memory doesn't exist solely to support swap files. It also allows any program to assume it's the only thing running (which simplifies assembly code) and enables memory-mapping of files and devices.

(Yes, you'll still need to memory-map files when we have memristors. They still won't necessarily be contiguous on disk, and they still won't necessarily exist locally.)

Probably has to do with the way we interact with RAM (memory addresses?) vs persistent memory (file system). Can those two be merged, or is the whole system like a giant RAM disk?

> Fink has assigned one team to develop the open-source Machine OS, which will assume the availability of a high-speed, constant memory store. Another team is working on a stripped-down version of Linux with similar aims; another team is working on an Android version, looking to a point at which the technology could trickle down to PCs and smartphones.

I was hoping for a second there that somebody at HP had read about Loper OS and Stanislav Datsovskiy's ideas[1], or even about Symbolics LISP machines (great article on the frontpage today[2]), but nope, looks like they are just gonna repeat the same insanity of the past fifty years instead of actually doing something that will progress the field of computing.

[1] http://www.loper-os.org/?p=284

[2] https://news.ycombinator.com/item?id=7878679

I guess it would be like an OS that once loads and starts, will run forever. You can put your machine to sleep which will freeze the sate, and then bring it back on again and start processing instructions from where it left off.

But if that's the case then they'll have to think of ways to do e.g. OS upgrades - where you normally reboot to flush the old code out of RAM.

Or how will you recover if you load a bad driver end experience a blue screen? Normally you reboot to flush the bad state and start again.

Somehow I think you'll need a separate cut down immutable OS alongside the your System so that you can repair things or make updates to the main RAM/Storage space.

Or at least that's how I understand it..?

I really don't understand this whole discussion about redesigning operating systems for this. Just because you have a machine that doesn't lose memory state on power failure doesn't mean you can't reboot it.

It means something like this has never done before, and so HP would be the first to develop the software to do this!

Reply on news.ycombinator.com