So perhaps the real solution is to provide some kind of mechanism for userspace processes that act like kernel drivers, in that they are definitely running all the time, but actually run in userspace.
So perhaps the real solution is to provide some kind of mechanism for userspace processes that act like kernel drivers, in that they are definitely running all the time, but actually run in userspace.
The last is something you need kernel help to enforce.
Most programs are dealing with vastly slower bottlenecks. Like network or disk latency. IPC latency is so far down the list of priorities in terms of things to optimize.
Not to mention that regular user-kernel-user buffer copying is plenty fast now, especially since the buffers will likely be cache-resident. Zero-copy is mostly hype without performance numbers to back it up. Don't need this complex message sealing mechanism.
The reason this is in the kernel is that DBus is slow (according to Linus, because it could have been written better by monkeys with typewriters), and Lennart wanted it in the kernel/systemd, and Greg KH agreed, and Linus trusts Greg.
https://lists.linuxfoundation.org/pipermail/ksummit-discuss/...
Unless the application is a HPC parallel computation that is IPC bound, Bus1 has no performance justification. If the main justification for a modern kernel IPC mechanism is performance, I'd much much much more trust a proposal coming out of the HPC community than the desktop experience community.
So now you have two ends of the spectrum where this effort does matter, performance wise.
This only leaves the middle (desktop), but there are other reasons why that evil desktop crowd wants IPC in kernel, however high on a ‘priority list’ they might be.
You have wifi, screen brightness, CPU-hungry background apps, etc. to fix first.
It’s not about tight loops.
Also this idea that you have to fix wifi chip or screen before you can add anything to the kernel doesn’t seem to be founded in any kind of logic.
For priority inversion it seems dbus-daemon is at fault here, not user-space message routing in general. It's just as easy to have two message routing processes, one for low priority messages and one for high priority messages.
As far as 4 context switches versus 2 with regards to latency. A context switch takes microseconds... This is vastly smaller than the 250ms UI response time.
Again, this looks like a solution looking for a problem.
Fixing wifi or screen before working on micro-problems in the kernel for the sake of global power efficiency is founded in every kind of logic. It's literally Amdahl's Law: reducing power usage in a component that already has the most minimal power usage is a waste of effort for no noticeable effect. The cost-benefit ratio is indefensible. This is insanity.
Linux is lacking a coherent IPC system and the necessary primitives to build one. That’s the bottom line.
Please, I can argue down any allegedly rationale for creating Bus1. Try me. You have yet to put forth any convincing argument.
The real bottom line is Linux already has a coherent IPC system but third parties don't like it and are irrationally fixated with an in-kernel solution that reflects their grand idea of what a coherent IPC system should look like. It's childish, it's bad engineering, and it's a waste of effort.
* for priority inheritance, have multiple daemons for each message priority level. Only privileges processes can use the high-priority message router, but can optionally use the low-priority message router when they want to inherit the priority of their sender.
* like I said, context switching is irrelevant since context switching is not the bottleneck in event latency nor in power usage.
* if FD accounting is important, move it out of the broker and make it p2p with broker coordination.
* kernel and LSM hooking feels like a feature looking for a use-case. If anyone really wants to do that, I'm sure the specific use case can be accommodated. Not every user space program need be trackable via LSM.
* for side channel and message ordering, why is this a core feature of the transport layer? I don't think all applications need this and if they do, something simpler and more specific to the application protocol could be developed. Generalized message ordering guarantees as part of the transport layer seems like it violates end-to-end design. A robust application protocol will end up ensuring this on its own anyway.
There you have it. I'm sorry but you don't have a strong case. Linus is even more opposed to this than I am and is probably better about system design than I am as well.
Obviously, if you do not care about any of the features, then you should not be using bus1. But that does not mean that others don't care, so why are you objecting so strongly to us providing a solution to these problems?
Firstly, if you don't want a bus IPC with multicast messaging, then use something else. But if you DO want that, then you cannot ignore message ordering, that is the whole point (otherwise you could just use repeated unicast instead).
If you want message ordering you cannot split things into several daemons, or you break the order (unless you add some more communication on top). Same with p2p (side-channel) communication for FDs.
The broker is basically doing work on behalf of the sender, but without proper time-slice accounting it is not accounted accordingly. Obviously in many cases that does not matter, but you cannot simply ignore the problem if you want a general, lasting solution.
It is true that performance is not the main feature, but that is not to say that it does not matter at all (to anyone, ever), so you cannot simply brush it aside, proposing a solution that would have (almost by definition) hundreds of percent worse performance (as you are just doing more round-trips, even ignoring the context switching). To say that IPC latency is "not the bottle neck" only makes sense if you have a specific use-case in mind. With sufficiently low latency, we open up the possibility of using IPC for things that could only be done in-process today. This is not the MAIN purpose, but ignoring it is short-sighted.
I just got an email today from people asking about the LSM hooks, so there certainly is interest, and this (like the message order) is not something you can bolt on top.
I have not seen Linus comment on this, please give a link if you have. Whatever reservations he had about kdbus should not apply to bus1.
Look, there are plenty of things that could be improved on bus1, and I'm sure some careful review will find things we could and should do differently, but please at least spend more than five seconds looking into what we did and thinking about the problem. We want a multicast IPC system, but we want one done properly. You cannot simply say that doing one that is several times slower, doesn't do proper resource accounting, gets the message order wrong and cannot integrate with LSM is "good enough for me" and expect that to be the end of it.
Multicast messaging as your main motivation for a message ordering layer when that layer has drastic effects on performance and operational characteristics is a strange proposition. I'm willing to argue that 99% of application protocols will have no use for baked-in multicast as you've implemented it. What exactly is the killer use case for multicast messaging with side-channel aware causal message ordering? What prevents an application that cares about this from ordering incoming messages according to a lamport clock itself?
I see no downside to these features in general, it's just that the justification for putting it in the kernel is weak. Kernel code has an infinitely high maintenance cost and it's not clear bus1 is the end-all messaging layer you want it to be. There are lots of past failed examples, inotify, dnotify, and fanotify come to mind...
Finally, He said what we're all thinking!
> o scale with the number of CPUs available [...]
It seems to me that these two goals are conflicting. If they give up the first goal then suddenly the need of a broker disappear.
edit: quoting from the message linked by parent.
edit2: by reading the description of the ordering guarantees, it seems that they actually only guarantee partial ordering.
There is no partial ordering between proc1->proc2 and proc3->proc4 as the memory channels are invisible to bus1, but Proc2 and proc4 can detect the total ordering violation by using another counter in shared memory.