Hacker News Reader

Comment by ksec | Hacker News Reader

So we could fit, 512GB DIMM, at up to 8 TB of Optane, and using them like in~memory DB with persistence? I wonder what performance improvement would I get on a Postgre DB compare to RAM + NVMe SSD.

View on HN

A lot of the overhead there might come from Postgres (e.g. building temporary in-memory indices for hash-joins, because it's assuming random access to the tablespace is slow.) Ideally you have a DBMS that already understands that it's running on nonvolatile memory, and so doesn't have separate "in memory" and "on disk" formats for its data.

Maybe something like Aerospike? (don't know much about it but I've heard it's "for" that)

Or some LMDB-backed database, sure. With LMDB you'd essentially map the entire address space of the disk, then just persist pointers. Writes would still not be as performant as you would hope, but random read performance would be in the small handfuls of nanoseconds per.

Running Redis would be pretty fun on this.

Just running Postgres on an NVDIMM, or any other database for that matter, will give you a large speedup compared to normal storage, especially when you have strong durability requirements for a transaction (fsync() will be faster).

But to fully benefit from persistent memory, the DBMS will need to modified. To see how that might look, read this [0] post by Microsoft about their efforts in SQL Server.

There's also an interesting research paper from CMU [1] that talks about challenges associated with pmem in the context of databases.

[0] - https://blogs.msdn.microsoft.com/bobsql/2016/11/08/how-it-wo...

[1] - https://www.cs.cmu.edu/~jarulraj/papers/2015.storage.sigmod....

Reply on news.ycombinator.com