cycloptic

That's what I mean though, I see the theme, but it seems to me to be about the same as trying to fit everything into an HTTP REST API, it all falls apart when something comes along that breaks the abstraction. For example if you have something that wants to pass a structure of pointers into the kernel, you can't reasonably do that with 9p, so now you've got a special case. The debug APIs can still only return a direct memory mapped pointer to the process memory as a special case, the normal case is doing copies of memory regions over the socket, no matter how large they are. If you want to add compression to your VNC thing, or add some more complex routing to your network setup, you have to start adding special daemons and proxies and translation layers into another socket, which is not really different from what you would be doing on a more traditional Unix. Or is there another way plan9 handles these?

Ok I see, that helps, thank you. That seems to be mostly similar to evdev on Linux after all, except it requires you to use coroutines instead of having an option for a poll/select type interface.

To me the problem with saying "no special cases" seems to make it quite limited on the kernel side and prevent optimization opportunities. For example if you look at the file node vtables on Linux [0] and FreeBSD [1] there are quite a lot of other functions there that don't fit in 9p. So you lose out on all that stuff if you try to fit everything into a 9p server or a FUSE filesystem or something else of that nature.

[0]: https://elixir.bootlin.com/linux/v5.11.8/source/include/linu...

[1]: https://github.com/freebsd/freebsd-src/blob/master/sys/kern/...

How is that conceptually different from IPC? The graphics system appears to somehow pass mouse and keyboard events to the client programs over a channel. At least that part seems similar to an Unix X11 setup where this would be done over a socket.

I guess I just don't see what is conceptually the difference here versus something like doing basic HTTP over a TCP socket, it seems like the same kind of multiplexing. Either way, you still have to deal with the same issues: can't pass pointers directly, need to implement byte swapping, need another serialization library if you want the format to be JSON/XML or if you want a schema, etc... So in cases where that stuff isn't important, channels would come in handy, but of course that is now getting closer to a local Unix IPC. Am I getting this right?

>No more broken than mmap of nfs.

Right, I get that's what you meant, it doesn't seem to really change much versus NFS, or DCOM, or whatever. So it's unclear what benefit is being provided by 9p here.

Also upon further research I am not sure what you mean by this is the only option, plan9 seems to suggest use of channels for other types of IPC interfaces, which seem to not be the same as 9p and are not necessarily network serializable. (Or are they?)

>there's no other option for accessing resources

That seems like it would create difficulties in porting software there. Please correct me if I'm wrong but the original plan9 appears to also have no support for shared memory or for poll/select.

>Backfilling IO on page fault is really all mmap does, conceptually.

For read-only resources yes, for handling writes to the mmapped region, that seems quite broken.

>You no longer have to wonder what happens if you try to create a Unix socket on an NFS file system, and then mmap it

How does that work? I don't know the details of any implementation, but 9p the protocol appears not to have any concept of mmap: https://9fans.github.io/plan9port/man/man9/intro.html

I think I see what you mean about 9p not being that special, it doesn't seem much different than if Windows decided to export every system-level API as a DCOM object, that would also get you the same kind of "the whole universe is networked" kind of deal.

I'm guessing these applications do not have any kind of animations or smooth scrolling? That would be a simple test, make your web browser or your image viewer fullscreen in 4K and see if there is lag in the scrolling/panning/zooming.

Yes -- so if the company falls behind, maybe using some open source code could help you keep up. At least that's what I've felt looking at things on github/gitlab/etc has helped with, if done the right way :)

Then that sounds like you would want it to be an optional plugin, or a compile time flag that could be enabled for customers who want it? Why not do it, if that's what the customers asked for?

I get what you are saying and it definitely applies to the core architecture of the product, but if you have a large number of customers each with their own needs then I would be surprised if there were zero parts of the product that were modular or interchangeable.

I am not sure I see what the significant difference is, I've heard of security escapes happening in both Docker and in various hypervisors. Either way there is a risk of some privilege escalation bug that allows access to the full RAM. I think if you want isolation, both of them lose out to having a separate firewalled off machine. Also I think the companies running heavy Linux workloads on Docker are probably not interested in the ability of Qubes to run Windows or Mac, just my read on the situation from talking to some of them.

I don't know about snap but from what I have seen of flatpak, it allows for different versions of the same package to be installed, not many package managers are supporting that currently. (Nix and Guix being some notable exceptions, and those should be able to re-use some of the sandboxing bits from flatpak if they need to) Of course, that is one of the main benefits to building this on top of filesystem overlays, and why it requires a different approach from a traditional package manager, i.e. it's not just a duplication.

Edit: Live migration actually does work for containers, take a look at CRIU. (I don't know the current status of this being integrated in Docker) I never even saw this as being opposing technology anyway, for example if you need to you could migrate a container in or out of a VM.

I don't see what you mean. Qubes is great, but it is not the same thing as Docker, flatpak, or snap. Are you saying Qubes should somehow be changed so that it works similar to Docker? And if so, why wouldn't you just use Docker?

I am not saying it would be easy, it would definitely be work that someone would have to do. I'm more saying that the Godot authors would not try to threaten you or consider you a threat, it's more likely they would want to help you. Of course as a business you would not do it if your customers weren't going to pay the cost to make it worth it.

I don't know enough about the implementation of C# and Swift to say for sure, but it does seem like the open sourcing of that would making it easier to do things like port some standard library component or algorithm over from one to the other, or perhaps do something like building a Swift implementation for the CLR.

I still don't see what you mean. It sounds like you are saying the real threat would be if they had no other features that could let them stand out in the market, at which point a competitor would be able to beat them by lowering the price, possibly to zero. That can be done by any competitor and has very to do with the license -- my point is that the open source license on that "competing product" actually helps them, by allowing them to make use of the same thing without having to pay that initial investment again. And the first initial investment you made isn't lost as long as you keep a path to retaining those customers.

To put it another way, if the actual problem to the business is that they are falling behind on feature velocity and don't have the head count to keep up, re-using some features from open source code could actually help there.

I don't see how it is a threat. Assuming Godot obsoleted all their code entirely, that would still be a boon -- that's now code they don't have to spend time maintaining anymore, and they can just reuse that and focus on their core competency. (Maybe it's hosting, I don't know enough about this business)

Different languages actually isn't as bad an issue with this type of thing, as the idea with running it in the browser is that it all compiles down to Javascript or WASM.

Well, of course it would take work to compose them together, but then the pay off is that you might be able to say customers are getting the "best of both worlds."

If their customers are also asking them for hosted Godot, maybe they should also offer that as another product offering, at a competitive price, and then use that as a sales funnel into their other products? That is usually the way it goes with these open source bits.

I am not sure why you would feel threatened even if there was overlap. Godot is MIT licensed, so if your customers started asking for features from Godot or for compatibility with Godot, you could just copy the code straight from them without any hassle.

From that perspective, moving to OpenBSD seems mostly pointless as currently the best practice there if file permissions are too strict seems to be "comment out some unveil lines and recompile the program." Not really an improvement IMO.

From that angle if the permission dialogs bothered you then you could just recompile flatpak to unconditionally approve all dialogs. (Maybe there is even a setting for this already?) Of course as a sibling comment has said, this would be pretty dangerous, almost equivalent to using windows without UAC, or sudo with NOPASSWD.

The safest way to do it would be to implement it with seccomp so you unconditionally block those syscalls.

>If you're expecting a blocking system call, and actually get a brand new background thread that's polling, it's quite reasonable to be frustrated.

It really isn't if the documentation doesn't outright say that it's single threaded and not thread safe. For a lot of simpler use cases where you just want to ship a thread-safe API (e.g. application does not have its own thread pool) then it just makes sense in a lot of cases to use some kind of automatic thread pooling. The caller does not have to know or care how the internal state machine is implemented.

If you have implemented your own thread pool it seems you should know enough to dig down enough to the lower layers to where you can get to that blocking syscall, or least to the point where you can strip off the O_NONBLOCK flags yourself.

The systems that use it as a native threading model are obsolete, but there's also this sentence there:

>Cooperative multitasking is used with await in languages with a single-threaded event-loop in their runtime, like JavaScript or Python.

There's no reason rust can't have an executor that does the same, and you only use that within the event loop on your one or two HTTP worker threads. If you're waiting in a thread for an HTTP request to return, that's never going to be CPU-bound. I still am failing to see what the problem here is besides a complaint about some rust crate only supporting a multi-threaded executor, which again is a different problem than whether it's done with async futures or not. One could just as easily write some C code that forces the use of threads.

Not really? That's cooperative multitasking and it's used a lot: https://en.wikipedia.org/wiki/Cooperative_multitasking

But regardless, the GP post was not taking about matrix math, it seems it was talking about sending an HTTP request and waiting for a response, which is something that actually is I/O bound on the TCP socket.

In those cases it would be the company that is volunteering, the contribution is still voluntary, and the company is paying someone to volunteer on their behalf.

Why not? The fix there sounds like it's as simple as adding some yield points.

That doesn't seem to be related to async? I don't know the details of rust's async implementation but that sounds like a problem with your application's setup -- you should be able to have a single threaded async executor that uses an event loop, or in simple cases, just calls poll/select directly?

To put it another way, it's unfortunate that particular synchronous API is implemented using threads, but there is nothing about async that implies one way or another that a synchronous method will be implemented using threads -- I've seen plenty of (questionable) C functions that do similar things like using pthread_create and then pthread_join immediately after to fake a blocking task.

I am not sure what the problem there is, that is a patch that carries a non open source license, which is allowed by the original ISC license of pgbouncer.

To the contrary, if Amazon is providing a good hiring funnel for the developers/maintainers, regularly contributing patches back upstream, providing funding to the project's non-profit, and generally respecting the license, then what's the problem? I'm no fan of Amazon but how can I complain about them having a right to profit in cases where they actually are being good open source citizens? Are they really any different from any other cloud provider in that respect?

I think we would all love it if we could magically fix every bug or design flaw in existence with a wave of the hand, unfortunately in real life it takes time and experience to do things right.

I don't understand what you are saying about freedesktop.org. That is just another volunteer run open source organization that hosts projects that are loosely related to open source desktops, you can start contributing to that if you want, or you can not use any of it if you don't find it useful. I'm just here for an interesting conversation, I'm not trying to convince you of anything, and I would rather not continue this discussion if you're going to start throwing around insults and making it personal and accusing other developers of being ignorant or having bad intentions. Please don't do any more of that, it's not interesting conversation and it's against the rules here. You're better than that. If that's tone policing then I'm sorry but my point is we ultimately can't have a conversation if your goal is to attack other people who aren't even here and tear them down, that just isn't my goal.

I also still don't see what you mean about dbus, the Linux kernel itself doesn't specify a wire format for arbitrary messages, and doesn't specify all the things that you need to get the complete functionality of a message bus. Maybe you could get that with another operating system that is based around message passing but Linux is not that. The methods you describe could technically be done without a daemon, but they still require a lot of additional code to set up a bunch of files and sockets and enforce ordering, security, etc, which could also contain bugs. You could tell the applications to implement all that themselves or you could put it all in a daemon which is mostly what dbus does anyway, and by doing it in one daemon it totally eliminates a certain class of race conditions and synchronization issues. Again please refer to the comments by the dbus developer that I showed, this conversation is not new and already happened years ago. If you want to store ASN.1 or S-expressions in a d-bus message you can do that pretty easily. And if you really believe that your solution could work then I would encourage you to develop a dbus implementation that works like you describe and then test to see if it works exactly the same and doesn't break existing setups. But I don't think this would really work, you wouldn't really be saving many lines of code, and in particular multicast and service activation would be pretty hard to do in the way dbus does it without a central message bus.

If you don't agree with GNOME or KDE's policies and you want to implement your own IPC then that's great, I support you doing what you need to do, however they chose dbus a long time ago, and currently it's looking like X is not the right tool for them anymore, so you may just have to accept your differences and move on.

If you're talking about making something that is Linux only, that also would be a no-go for X clients that expect to run on other operating systems. They would still want to maintain the old code path, so it wouldn't help much.

You do have to bring your own multiplexing in the X world if you were implementing an X compositor. Maybe that's unfortunate to you that the focus changed to X compositors, but Wayland didn't change the fact that this work has to be done by some willing party to get that to work.

I wouldn't call myself a "Wayland fan" but what you are saying isn't really a problem, the different window managers can choose to implement it just one way. They don't have to do it N different ways, of course they will do it differently if they have a valid reason to. From an application developer perspective you shouldn't have to deal with this problem, I'm sorry if you are a toolkit developer and this has caused you pain, but in my experience nearly all of the bits that you would need to have to do a native port to Wayland aren't specific to any window manager.

Not really, shared functionality there is in the core wayland protocol and in the standard set of wayland extensions, or in some other standard dbus interface. (generic ones usually go in the org.freedesktop namespace) If you depend on GNOME-specific or KDE-specific APIs then yes, you would have to deal with those being unsupported outside. Not much has changed there.