Can someone please explain what's wrong with the classical UNIX sockets and pipes that they need to invent and (re)invent dbus, bonobo, now bus1?
But this paper makes a good quantitative case for POSIX not meeting the needs of modern computing: https://news.ycombinator.com/item?id=11652609
A good example is that Linux distros, Android, and OS X all have their own non-POSIX IPC mechanisms. Applications are somehow voting with their feet and not using Unix domain sockets and pipes.
It leaves me scratching my head because I don't think serialization is that hard a problem. But it seems that most people want IPC with structured data. Android is a bit of a special case because it's a single-language OS.
"Traditional POSIX threading models, IPC interfaces, and file system access are being replaced by platform and vendor-specific APIs and frameworks such as Grand Central Dispatch [18], Binder [29], DBus [25], and SQLite [1]."
Specially when the majority nowadays has the idea that GNU/Linux == UNIX and never tried to write an actual portable UNIX application.
In fact, Mac OS X is the most widely deployed UNIX-like system ever. What you're a claiming to be true probably more applies to Mac OS X than Linux.
I honestly haven't seen a recent package that was accidentally not portable in the long-term across the various modern UNIX-like systems because of confusion of Linux for POSIX.
I don't miss my days of writing portable POSIX code across DG/UX, Solaris, HP-UX, Aix, FreeBSD, Red-Hat, Debian and the respective #ifdefs.
You were asserting that the majority of programmers think Linux == UNIX and have never tried to write a portable application.
Thst is clearly false as the majority of programmers probably confuse Mac OS X for UNIX and most modern unix-like software is portable to at least Linux, Mac OS X, and FreeBSD. Portable is more general than "portable across every UNIX that has existed."
Increasing the specificity of your argument mid-way to avoid the incorrectness of your initial general argument isn't very genuine.
But yes, your revised second statement is probably correct: most UNIX programmers have never tried writing portable code across "DG/UX, Solaris, HP-UX, Aix, FreeBSD, Red-Hat, Debian" exactly because there is no practical reason for it.
It is a hard problem, at least if you want to achieve concision, make it parseable, and support schema upgrade.
> "Traditional POSIX threading models, IPC interfaces, and file system access are being replaced by platform and vendor-specific APIs and frameworks such as Grand Central Dispatch [18], Binder [29], DBus [25], and SQLite [1]."
And good riddance. File systems are almost universally terrible [1]. Threading with shared state is also a terrible idea because it's too general, not to mention the poor APIs used to manage them.
IPC is something that's still unfortunately undergoing constant revision, and the structured binary vs. text wars unfortunately both have merit. I think structured text output via a domain-specific language (DSL) is really the proper answer, somewhat like how a REPL works. This DSL is ultimately simple to construct within C since it would be lisp-like in its structure so it doesn't need an IDL compiler, although its textual syntaz shouldn't be lisp-like.
stdin can then nicely format DSL output for the user. Best of all worlds between text, binary, and structured formats.
Yes and no. There's significant utility to standardizing serialization in your IPC layer. IPC consists of exchanging domain-specific messages, so the message types can be specified in a header shared between endpoints, and data gets serialized in a standard format.
The IPC API then simply reduces to sending/receiving data to/from endpoints, like channels in CPS. And ideally it should be higher-order so you can send an endpoint to another endpoint.
So you don't need to standardize the serialization, but then you just get a plethora of incompatible formats.
You can't pass file descriptors over sockets. You can't do RPC over pipes. Anonymous sockets can only be shared with children. Named sockets require write permission and a filesystem. SysV is prone to resource leaks.
Binder has authentication, shared memory, reference counting, weak references, dead object notifications.
The descriptors and authentication part is key, as it allows Android's trusted system_server to open a device or socket for an untrusted process such as an App if it has user permissions. The receiver of a binder request can check the requester's PID/GID and with confidence from the kernel.
Most importantly, I don't think it's healthy to think of POSIX as being done. Some experimentation and discovery of better APIs and better systems should be encouraged.
gah, I'm driving myself crazy; I swear I saw somewhere a diagram of the Taxonomy of Unix IPC Mechanisms, with most of them crossed out and replaced simply with binder, please help me find it!
Good that my code doesn't know this. It would stop working.
> The descriptors and authentication part is key, [...]
If by "authentication" you mean "checking PID/UID/GID", we already have both in unix sockets.
If you mean "I send some bytes that describes the file to open, and keep my processes in sync manually", of course you can do that, but true passing would allow a number of cool things.
You can:
http://man7.org/tlpi/code/online/dist/sockets/scm_rights_sen...
I've seen that done several times, with defined request and response message formats.
Some call-outs for you:
> * efficiently support n-to-n communication. * be well-suited for both unicast and multicast messages. * guarantee a global message order. * avoid any in-kernel buffering and rather transfer data directly from a sender into the receiver's mappable queue (single-copy).
The downside of the alternatives that you mention is that they're not available as kernel features in Linux. Bus1 is inspired by that previous work and aims to make a new attempt at inclusion in the kernel, which they believe is required to provide the features they're targeting:
> A user-space implementation of bus1 (or even any bus-based IPC) was considered, but was found to have several seemingly unavoidable issues.
> o To guarantee reliable, global message ordering including multicasts, as well as to provide reliable capabilities, a bus-broker is required. In other words, the current linux syscall API is not sufficient to implement the design as described above in an efficient way without a dedicated, trusted, privileged process that manages the bus and routes messages between the peers. (...)
> Large automotive companies have built huge systems on top of QNX
> messages, creating large libraries used by their applications.
> They would like to be able to use those libraries
> on Linux, but often don't know that there is
> a way to get the QNX message API for Linux. It is called SIMPL
> and it works well.
http://www.qnx.com/news/pr_2471_1.html
And now I notice it was almost ten years ago! Nevertheless, too little too late...
AF_INET sockets don't do FD passing or credentials.
There was a patch to add an AF_BUS socket family some years ago that sounded like a pretty general solution but the network maintainer said "over my dead body".
Well, for starters they're a bolt-on that doesn't fit with the actual classic Unix philosophy.
2) They want the ability to insert a broker that manages permissions among the peers, but have the kernel enforce those permissions on a per message basis so the broker doesn't become a bottleneck. That's not easy to with the filesystem.
I mean, strictly speaking it's theoretically possible using the file system considering that in practice the only real broker would be systemd with a fixed set of policies, so theoretically the traditional Unix model of user, group, and supplementary groups would suffice by assigning users and processes into the proper groups, and binding message files into protected group-readable directories. But for whatever pretense, people want the solution to implement a system of opaque, ephemeral identifiers/capabilities.
It sort of reminds me of SMF, launchd, systemd, and similar changes. For example, I was installing Solaris 11.3 on a VM yesterday and was trying to figure out how to set the hostname. You used to set it by storing the hostname to /etc/nodename. Now it's stored in some opaque tree data structure deep in the bowels of SMF, which you set with some long and impenetrable command-line utility invocation. You can still set /etc/nodename; if no hostname is configured then SMF will initialize the hostname from /etc/nodename and delete the file. It boggles the mind.
* Inotify events are coalesced. Great for notifications, may be bad for discrete messages.
* The sender may want information about # of messages sent, synchronous delivery, etc.
Ergo, POSIX wasn't the end.