pbalcer

Disclaimer: I work at Intel on PMDK (pmem.io)

Both Windows and Linux implement DAX, which, as @the8472 explained, allows bypassing page cache in memory mapped I/O. Additionally, DAX optionally allows you to flush your data directly from user-space instead of calling msync.

And that's the gist of NVM programming model [0], its entire point is to allow applications to avoid the now hugely excessive abstraction layer of traditional storage.

And I will freely admit that programming to raw memory mapped files can be difficult, but there is ongoing work on making it easier. An example of that is, excuse the shameless plug, Persistent Memory Development Kit [1], which makes writing new software for this new type of memory much simpler.

Performance of an NVDIMM is obviously hardware dependent, but the now widely accepted programming model works with the assumption that persistent memory is fast enough so that it is reasonable to stall a CPU while an instruction is accessing it. I'm not sure on what hardware evaluations you are basing your claims on, but let me assure you that the HW solution being described in the blog post does not violate that assumption.

[0] - https://www.snia.org/tech_activities/standards/curr_standard...

[1] - http://pmem.io/

Yes, but to accomplish that you would have to use the MAP_FIXED flag, which is quite dangerous because it can replace previous mappings. That can lead to problems with dynamic memory allocation since almost all malloc() implementations use anonymous mmap.

That's an excellent observation. You are indeed correct that pointers in memory mapped files are quite tricky to get right. When you think about shared memory in general, this isn't a new problem [0], and the solution is almost exactly the same [1]. Instead of dealing with raw pointers, the library provides an encapsulated fat pointer which contains an offset from the beginning of the mapping. And when the file is opened, we simply register the new virtual address, and calculate the real address when needed.

[0] - https://www.boost.org/doc/libs/1_63_0/doc/html/interprocess/...

[1] - http://pmem.io/pmdk/cpp_obj/master/cpp_html/classpmem_1_1obj...

Yup. In this aspect libpmemobj could be compared to how Cap'N Proto [0] works. And of course, this has some trade-offs that users need to be aware of.

[0] - https://capnproto.org/index.html

The product being announced is for data centers.

NVDIMMs allow for a significantly different storage layer compared to a traditional block-based device [0]. This is mostly because it's on the memory bus, thus significantly reducing latency. See slide 8 in this presentation [1].

[0] - https://news.ycombinator.com/item?id=17195018

[1] - https://www.openfabrics.org/images/2018workshop/presentation...

Just running Postgres on an NVDIMM, or any other database for that matter, will give you a large speedup compared to normal storage, especially when you have strong durability requirements for a transaction (fsync() will be faster).

But to fully benefit from persistent memory, the DBMS will need to modified. To see how that might look, read this [0] post by Microsoft about their efforts in SQL Server.

There's also an interesting research paper from CMU [1] that talks about challenges associated with pmem in the context of databases.

[0] - https://blogs.msdn.microsoft.com/bobsql/2016/11/08/how-it-wo...

[1] - https://www.cs.cmu.edu/~jarulraj/papers/2015.storage.sigmod....

Disclaimer: I work at Intel on PMDK (pmem.io)

There's been a lot of interesting research around file systems for persistent memory.

One that shows a lot of promise is NOVA [0]. Its focus is on making full us on this new type of memory. And it's not just pure research, they are attempting to get NOVA included into Linux kernel [1, 2].

And while talking about file systems, we shouldn't forget about the effort that was put into modifying the existing ones to support DAX (Direct Access) [3, 4].

[0] - http://nvsl.ucsd.edu/index.php?path=projects/nova

[1] - https://lwn.net/Articles/729812/

[2] - https://lkml.org/lkml/2017/8/5/188

[3] - https://lwn.net/Articles/717953/

[4] - https://lwn.net/Articles/731706/

Disclaimer: I work at Intel on PMDK (pmem.io)

It is the game changer that you wish for, since the marshaling logic that you mention is gone. Persistent Memory can be accessed directly through memory mapped file, bypassing the traditional read()/write() I/O paths. Recent file systems have also been modified to a) skip the page cache layer and b) forgo the msync() call that would be otherwise required to synchronize the modified pages. This is what's called DAX (Direct Access [0]). In the place of msync() you can now just use CPU cache flush instructions. These two file system changes entirely eliminate kernel code from the I/O path (apart from the initial page faults).

Persistent Memory Development Kit contains libpmemobj [1], which is almost exactly what you are imagining ;) It's a persistent heap, with transactions for durability. It's not as nice (yet) as your code snippet, but here's C++ example [2] of a persistent queue push:

  obj::transaction::exec_tx(pool, [this, &value] {
    auto n = obj::make_persistent<Node>(value, nullptr);

    if (head == nullptr) {
      head = tail = n;
    } else {
      tail->next = n;
      tail = n;
    }
  });

`make_persistent` is, akin to `make_unique`, a memory allocation of a "Node" class. Once allocated, we can just assign the newly allocated object to a different persistent variable. No kernel code executing, no serialization ;)

[0]- https://www.kernel.org/doc/Documentation/filesystems/dax.txt

[1] - https://github.com/pmem/pmdk

[2] - https://github.com/pmem/pmdk/blob/master/src/examples/libpme...