Hacker News Reader

Several of the folks I've talked to who were most excited about Optane (and disappointed about its delayed introduction & market rejection) were interested primarily in the greater per-chip density, with persistence as at best a "nice-to-have".

Take your typical in-memory database: The primary scalability constraint is how much "fast enough" memory you can park close to your CPU cores. Persistence lets you make some faster / broader failure & recovery guarantees, but if those are major application concerns, you probably wouldn't be using an in-memory database in the first place.

>and disappointed about its delayed introduction & market rejection

Even at peak NAND and DRAM price ( at ~3x of normal ), where Optane had a slight chance of competing, Intel was still selling it at cost, with a volume that could not even fulfil their minimum purchase agreement with Micron. With a technical roadmap that doesn't fulfil their initial promise, and questionable cost reduction plan even at optimal volume.

Compared that to DRAM ( DDR5 / DDR6 and HMB ) and NAND ( SLC / Z-NAND ), as much as I love the XPoint technology the cost benefits just didn't make any sense once DRAM and NAND falls back to normal pricing. Micron already know this all the way back in 2018, but the agreement with Intel kept them from writing it out clearly.

I'm curious: The promise of Optane has pushed Intel's stock upwards. Did it's rejection by the market pushed the stock downward in a similar amount ?

I honestly dont think that was ever the case, unless it was some mainstream media reporting. In reality Intel has been making record profits YOY for 4 or 5 years in a row. Mostly due to Datacenter revenue. Optane, or NAND / Storage business is a tiny segment on Intel's profits. Although the narrative that Optane will create a lock in on Server Sector certainly helped a tiny bit. But now the threat is coming from elsewhere.

That’s a good point. If I remember correctly, optane has a operating mode where the memory is only meant to be used as RAM. It’s stored on the non-volatile RAM in an encrypted form and the key is intentionally discarded/lost when rebooted.

Another obvious operating mode is to make the NVRAM be the first line of swap and have things spill from DRAM into NVRAM when there is memory pressure. I think Intel also offered a mode like this, where it made everything look like DRAM and it would spill onto NVRAM as it needed to. Of course, this would be better rolled into the OS or application level so smarter decisions can be made, but given very high NVRAM densities it’s a great application for the technology.

I think technology like this should force us to rethink how we do stuff at a fundamental level. Does the concept of treating "storage hardware" and "compute hardware" even make sense in this world?

We should probably start rethinking everything we do from the OS layer to the way we write applications and exchange data. Truly performant non-volatile RAM would be (will be?) a revolution that rivals some of the earliest groundbreaking work in computer science.

Definitely. You also have stuff like putting compute into the hardware. I read somewhere about SSDs with on-board compute doing serialization/deserialization of some data structures on the device itself, for instance. You can, in principle, take compute on board a NIC and if you could address the memory controller directly, you could serve NVRAM backed key-value store directly, without the CPU at all. Things are getting mushy and our way of addressing hardware resources and the OS abstractions behind them aren’t up to the task anymore.

In the research world, due to various things about the kind of work that is feasible to do and get published in a reasonable timeframe, they just aren’t looking at these bigger questions as deeply as I think they ought to. People do more minor iterations on the existing approaches, which is a pity.

That's not putting compute into hardware (i.e. for the first time), that's just building custom application-specific hardware, which happens all the time. You used to have the floating point unit outside the CPU.

Some fundamentals don't change. One one side there are processors (CPU, GPU, whatever) and on the other side we want to persistent some data for decades in some format that lets us exchange it with other people or multiple applications inside the same box. I don't expect ext4 and NTFS to go away soon. In between could totally change.

Why would it be any different than now? What fundamentally changes if RAM itself became persistent vs. being persistable to other hardware, except you'd save the one-time data loading cost? Case in point, RTX 3090 GPU's have 24GB of RAM on board, and they're left always on.

I agree that nothing would change. Even the "flushed to journal" approach of databases and filesystems would remain, because that doesn't just happen because of the looming danger of power loss, it also happens because software malfunctions exist and you want to separate work in progress from the done and approved.

And if it's not cheaper as RAM it's not interesting anyways. Chances are if you could afford 1:1 paging plenty of applications would happily do and not mind the separation between a fast copy and a persistent copy at all.

If RAM and storage become one and the same the way you store your data in memory is how you store your data permanently. The state of your application becomes your data storage. Or at least in theory it could.

> The state of your application becomes your data storage.

That's honestly a worst-case scenario. Imagine an application that is permanently in RAM -- and no matter how often you kill it, it will never "reload" from a clean state.

There will likely be other ways of resetting the state of memory.

Why you are asked to "clear cache" so often on Android.

Erlang and kin rely explicitly on the ability to crash and start over with a clean slate.

Yeah but the point is you can do that today. The OS does it with hibernation.

Imo, what kills Optane, is that it has limited write cycles, like an SSD, and because of that you can't treat it like RAM.

If I'm reading this right, I think UltraRAM doesn't solve this problem. From the article:

> its fast switching speed and program-erase cycling endurance is "one hundred to one thousand times better than flash."

I know flash can get some impressive endurance now, but if you start treating it like RAM, is 1000x that endurance even enough?

> I know flash can get some impressive endurance now, but if you start treating it like RAM, is 1000x that endurance even enough?

Anything with limited endurance will need some kind of controller in front of it that makes sure that it wears out evenly. You'll end up with "copy-on-write" RAM that will be inherently slow/space inefficient, either due to block size being way too large for typical RAM access pattern or the data structure managing it becoming too large.

And of course you incur extra latency from having to do look-ups in the first place.

This is a problem. When I worked on it I didn’t treat it as RAM, but as something RAM would spill to. Essentially treating it as where memory mapped files are stored and then being clever about how those files are read/written. This reduces writes to once per modified byte address within a transaction (e.g., between msyncs).

Another comment already mentions this, but I guess a functional data structure would be easiest to maintain, even though there's stuff like this for PMEM such that any data structure would work essentially: https://pmem.io/

This[1] Open Source data store I'm maintaining is storing a huge tree of tries eventually on durable storage, but it's essentially a huge persistent index stored in an append-only file. The last layer of "indirect index pages" stores list of pointers to page fragments, which store the actual records (the list however is bound to a predefinded size). A single read-write trx per resource syncs the new page fragments, which mainly only include the updated or new records plus the paths to the root page to a durable device during a postorder traversal of the in-memory log. The last thing is an atomic update, which sets a new offset to point to the new UberPage, the root of the storage. All pages are only word aligned except for RevisionRootPages, which are aligned to a multiple of 256 bytes. All pages are compressed and might in the future optionally be encrypted.

I guess these byte addressable persistent memory devices are an ideal fit for small sized parallel random reads of potentially tiny page fragments to reconstruct a full page in-memory.

However, I hope they'll eventually achieve a better price in comparison to DRAM and SSDs than Intel Optane DC PMEM current price range.

[1] https://github.com/sirixdb/sirix

All of that stuff is really interesting from a technical perspective, but if the persistent memory isn't cheaper than RAM then 95% of machines won't need it, or won't need more than a small buffer of it. It'll be amazing for database servers and almost nobody else.

And Optane failed that test.

I know of "persistent" data structures from dabbling in Clojure. Just thinking off the top of my head just from that little bit of knowledge. If the very last thing that happens in an update, is a pointer update that "commits" the new data, it's correct if we assume RAM is dependable and the pointer update is atomic. What if there was a command which updated the value in the persistent store, then coerced a cache miss upwards through all of the cache levels? If the bottom level update at the persistent store is atomic, then it doesn't matter if the propagating cache misses are interrupted by a power failure.

EDIT: But oh, the huge latency!

Ordering is what kills you there; lets say you have update X and Y both in cache and for correctness they must happen in that order. If you just do a global cache flush, you lose if Y happens before X. So you need to flush after X and after Y. This means that every update to a persistent store involves hitting memory twice. This also means that every implementation of a data structure needs to do all of this work correctly. Filesystems have many fewer data structures than general purpose code, and they still occasionally have correctness bugs here.

You might be able to salvage some performance by having a large write queue and running all cache as write-through, but that will make writes, on average, far more expensive than reads.

Perhaps a more reasonable option would be to use only static memory for the caches and have a battery-backed coprocessor flush the caches on power-loss.

Well said. I will just add that there are cache write-through instructions and fence instructions. You can use these for just the pointer swap components the parent was talking about, but absolutely you’re correct, you can just take a naive data structure and have this work out of the box.

There is work to make this transparent to existing data structures, but it (of course) imposes its own overhead.

Why not using DAX FS mode for Optane DC memory for instance and serialize the data structure by hand? I'm not sure how much performance you lose. That said the data store I'm working on serializes the huge persistent in-memory index to durable storage in a postorder traversal (mainly changed records in a data page Plus path copies) and atomically sets an offset to the main root, the UberPage to the new revision in a file. So, I'd say it's precisely what you describe :-)

At least on PPC you can map write-through, that solves 99% of this on an embedded system with battery backed SRAM I use.

Does it have unlimited write cycles?

No, it would last a decent bit if used ‘appropriately,’ but sadly in a heavy use environment it would be a consumable.