Hacker News Reader

Comment by jstimpfle | Hacker News Reader

> Files being persistent on storage has nothing to do with communicating through shared memory.

Files (whether persistent or not) have not really anything to do with communication through shared memory. In the implementation of an API like shm_open(), the VFS (virtual filesystem) is simply the address space and lookup mechanism that an operating system like Linux happens to use in order to find the memory that should be shared.

> It isn't necessary and it doesn't interfere if it's there.

Sure it does interfere. By backing memory needlessly with a persistent file, you're causing disk I/O from the loading and flushing (that can't really be controlled) and potentially bad performance.

Also, as explained, if you use a persistent file to track the synchronization state, the synchronization state won't be reset when the communicating processes die unexpectedly, and this might be problematic.

View on HN

system like Linux happens to use in order to find the memory that should be shared.

Right. Is there some other mechanism to coordinate mapping the same memory between processes? That's all I ever asked.

Sure it does interfere. By backing memory needlessly with a persistent file, you're causing disk I/O from the loading and flushing (that can't really be controlled) and potentially bad performance.

That is orthogonal, since once you have the memory mapped into both processes you can use atomics for lock free IPC. That's the whole thing. It doesn't matter what the OS does or doesn't do in the background, atomically reading and writing to memory is unaffected.

Why are you constantly ignoring what I'm writing?

> It doesn't matter what the OS does or doesn't do in the background, atomically reading and writing to memory is unaffected.

That's not true. If this thing is file backed there is usually no guarantee that the page of virtual memory (i.e. a page of the file data) you're accessing is present in physical memory. You'll cause page faults and data transfers to/from disk. This can delay the execution of an atomic read or write potentially infinitely, or even cause a "crash" of some kind if the disk transfer fails.

You can avoid the page faulting part of this if you somehow pin the memory. Which is completely ridiculous given that all you ever wanted is anonymous memory. I've looked up a website that seems to explain this better (but I haven't checked it deeply). Maybe it helps: https://eric-lo.gitbook.io/memory-mapped-io/pin-the-page

This can delay the execution of an atomic read or write

You can play "what if" all you want if you don't know what else running, but this was always about lock free interprocess communication, which is not broken by a page fault or process suspension.

An atomic instruction by design will do everything it needs to when the instruction runs.

Saying the OS can ultimately control the execution of a process is a nonsense cop out to try to skew away from the original point.

all you ever wanted is anonymous memory

This is local to a process tree and does not work for interprocess communication.

Dude, the example I gave you with shm_open() is creating anonymous memory. That's just what non-file-backed mappings are called, no matter how long you want to keep obsessing about any "file paths".

This doesn't even seem like a reply to what I said.

If you map memory anonymously you aren't doing interprocess communication.

If you don't, you have a file path that the other program can use to map the same memory.

That's it, there is nothing wrong with this. I don't know why this is so upsetting. Mapping memory anonymously is local to the process tree and doesn't work for two different programs communicating.

Ok, I'm extremely embarassed but it looks like I got the terminology wrong with regards to "Anonymous memory". And sorry for being so upset, at least I finally got something out of it.

It's also a fact that if I'm using disk swap space on a Unix, the same performance and stability issues apply as for disk backed file mappings. In that sense, there really is no difference.

Reply on news.ycombinator.com