Show HN: Xlibe – A serverless Xlib (X11) compatibility layer for Haiku | Hacker News Reader

Show HN: Xlibe – A serverless Xlib (X11) compatibility layer for Haiku

github.com

116 pointswaddlesplash46 comments

Hello HN,

While there are some projects that implement partial versions of Xlib APIs on non-X11 platforms for compatibility (most notably, Tk), I don't know of any others sophisticated enough to run Cairo and GTK applications (albeit with a few hacks). So I figured this might be technically interesting.

As of now it just implements the base Xlib APIs, it doesn't implement XRender, GLX, XInput2, etc. (though it's more than possible, I just didn't see a need yet.) Cross-program interaction is also very limited, and there are plenty of X11 features that likely can't be implemented; but this works sufficiently well that GTK3 is now in Haiku's default package repositories, with GIMP and Inkscape atop it to boot, using this.

And for the fun of it, I did try to compile Xnest (the X.org server variant running on top of Xlib) on top of this; it crashed on startup in the keyboard handlers. However, that could potentially be because I didn't have the necessary data files and not because of anything missing in Xlibe; I didn't investigate too far (or it could be an actual incompatibility; Xlibe's keyboard subsystem is rather primitive and is most of the reason I had to patch GTK for full functionality.)

(A few more screenshots, including GTK and WINE running atop Xlibe: https://www.haiku-os.org/blog/waddlesplash/2022-01-10_haiku_...)

Open articleView on HN

A few more technical details, for the interested:

* Every X11 "Window" gets its own BView (which is itself vaguely analogous to a QWidget in Qt, etc.) and of course top-level "Window"s attached to the root get their own "BWindow" as well.

* All drawing goes through the Haiku drawing APIs. (This does create some artifacts in applications that expect X11 drawing isn't antialiased, like most Xt/Xaw applications, but it works tolerably well.)

* Since Haiku has no "display server" / "windowing system" divide, all operations related to both are directly handled in Xlibe. This creates some trickiness when applications request "window manager bypasses" in tandem with modality management, and also in decorations/borders handling, but it works well enough.

And I forgot to mention, how silly of me: This work was largely done under sponsorship from Haiku, Inc. (https://www.haiku-inc.org/), the non-profit that manages the financial/legal aspects of Haiku development. If you find this and the other work I have been doing on Haiku in recent months good or interesting, please consider donating or setting up a recurring donation!

X11 API without an X server is a super cool concept. I followed the forum thread after OSnews reported on it late last year https://www.osnews.com/story/134362/xlibe-an-xlib-x11-compat... and your progress has been really exciting to watch. It's a great project. When Haiku got a Qt port it made a massive difference in software availability and this will too.

GTK will always owner-draw its buttons, so you don't get a real native Be widget. And it's clearly possible to do a good-enough X server integration as demonstrated by WSL, Quartz etc with minor complaints about clipboard syncing, decorations, having to run another daemon etc. After all that, what would you say is the main benefit to this approach?

Thanks for following along!

> GTK will always owner-draw its buttons, so you don't get a real native Be widget.

Well, I think it might be potentially possible to write a theme that uses Haiku control-drawing APIs, just like for Qt (already there are some CSS files that make GTK look like Haiku, but Haiku itself has a control-theming engine, so it would be good to look into that. But not something that's on my radar.)

> And it's clearly possible to do a good-enough X server integration as demonstrated by WSL, Quartz etc with minor complaints about clipboard syncing, decorations, having to run another daemon etc.

Having used X11+WSL and X11+forwarding quite extensively (and XQuartz a bit), my complaints about that approach are ... far from minor. Everything both looks and feels like it has been awkwardly shoehorned in. Window modality, sizing, and positioning always runs into problems.

Now, some of that is just MS Windows being Windows, and X11 not having a good way to deal with some of the differences between X11 models and Windows, etc. But I think a large part of it, having looked at X11 internals (and as you can see, knowing more than a bit about them in order to write this thing and make it work) is that distinctions which are more minor at the base levels of the API become significantly bigger the further up you go -- the window manager vs. display server is probably the most obvious one here.

Xlibe can largely forget there even is a "display server / window manager" distinction and take calls that deal with one or the other and invoke the necessary Haiku APIs.

> After all that, what would you say is the main benefit to this approach?

Besides the ones outlined above, I would say it's much more possible to integrate with the rest of the system in broad ways. For instance, clipboard handling can pretend to do the strange set/get property dance in Xlibe for compatibility's sake, and then turn around and invoke the (much more sane) Haiku clipboard APIs internally. With a real X11 server, the clipboard handling is just the standard get/set property system, and you have to have something else (in the X11 server) that forwards clipboard updates in both directions. That can quickly get very complicated.

Another thing is keyboard handling. I don't bother creating X11 keymap structures at all in Xlibe; applications either get XK_* values (for e.g. Return, etc.) or UTF-8 text (which can be locale-converted if needed.) That means you can swap keymaps in Haiku and Xlibe will follow right along. I have no idea how much code that would wind up being in an X11 server port to deal with, but I know for sure it would not be fun at all, having dealt with X11 keyboard APIs from more than one vantage point.

Drag and drop would probably be another, but I didn't implement that in Xlibe (yet?)

I haven’t followed BeOS stuff since the early oughts, but I’m floored at how mature Haiku has gotten. A modern GCC? Xlib mature enough to run GTK? A native QT backend? I also noticed what appears to be a well along Emacs backend as well. Amazing progress for an independent OS project.

The graphical Emacs backend, unlike the rest of the things you mention, was not developed by one of the "regular Haiku developers" but by one of the Emacs developers, and fully upstream, too, which makes it doubly exciting. Indeed it's now available in Haiku's package repositories (or of course you can build Emacs from source yourself.) :)

I hate to be the one who has to say this, but this is a bad idea. If this is being done for practical reasons then don't waste your time with this. It might work for some clients but you'll get stuck in the same place that every other re-implementation of X11 over the last 30 years got stuck at, which is supporting any kind of weird quirky behavior in Xlib and its large collection of client libraries, and the rest of the quirky behavior in the X server. Eventually you'll end up copying large sections of code out until it slowly transforms back into Xlib and Xorg and effectively it becomes another approximation of an X server. If that's fine with you then by all means keep going, but please don't continue along with the idea that you can avoid this. It has been tried many, many times before.

> If this is being done for practical reasons then don't waste your time with this.

Too late, I did, and it's already being used "in production" to power GTK on Haiku!

> It might work for some clients

Well, GTK was the one client I really cared about, and it works for that, so...

It should work for Tk, too, but there's one really annoying bug somewhere in Xlibe which is causing Tk problems; and I didn't figure out what it was in the Tk codebase (this is doubly strange because Tk itself has Xlib compatibility layers to run on other OSes, you would think they would have excised any such quirky behavior.) The "AndroWish" developer said he'd try to take a look and figure out what was going on, at some point.

> you'll get stuck in the same place that every other re-implementation of X11 got stuck at

So far as I've been able to tell, this is now the single most advanced Xlib compatibility layer ever written (well, that's publicly available and locateable, anyway. I wouldn't be surprised if there's more advanced ones behind closed doors.)

At least, I don't see any others powerful enough to run GTK, do you? So I think I've not gotten stuck at that same place then!

>Too late, I did

This reply makes no sense, I would greatly advise against falling for the sunk cost fallacy. If all you care about is GTK, then you could have just done a native port of GTK. That would actually be a lot easier than trying to reimplement every X client library. With GTK4 it should be even easier because the backends were simplified and a lot of the weird X idiosyncrasies were removed, and you might actually have a chance to get hardware acceleration to work without having to go through the pain of dealing with GLX. The problem with Xlib is that's is not just the one library, it's actually about 30 libraries. So like I was saying earlier, it's cool that you have one, but the hard and annoying part that isn't worth it is doing the other 29. Nobody has gotten that far, and if you do get that far, the best way to do it would be to copy a huge amount of code. So the goal here can't really be reached in any practical terms, unless you just want to stop here.

It isn't practical to write a port even for every major GTK version, much less for every other toolkit used with X11. Firefox for example is still on GTK3, and so are many other applications.

Haiku should want some form of generic compatibility, and this is one way to do it (I would have been lazy and used the embedded X server approach for it if I wanted to do it for some OS, but waddlesplash apparently has different tastes). Perhaps one day someone will want to do the work for a 'native' GTK4 port, but that does not degrade the important of this work for Haiku.

Using a hack of Xlib to port firefox is probably the worst possible approach, your rendering performance will be horrible and you still get to deal with the pain of the impedance mismatch from using a different API from what your system is designed to do.

I get the desire to take shortcuts to get things ported quickly but that will not actually help in making these applications properly usable.

Firefox was just one example. There's no practical way to port everything a user might desire - everyone has their own 90% as the saying/joke goes. Haiku needs a generic mechanism, and well, this is one way. Apparently waddlesplash likes this way, and is willing to invest doing the work. This is commendable and will help Haiku adoption. It's also technically interesting and cool enough for a HN post.

I will say that I don't think it's the best path for an alternative OS, and other AltOSs might wish to pick a different path. IMHO, Reimplementing POSIX is a mug's game, a lot of work, while Linux will always be ahead and POSIX isn't much good anyway. Microsoft showed the way with WSL2. This would be a better path for alt OSs in general:

A. Implement the OS with almost no compromises. Ignore POSIX/X11/etc. if they don't match the vision.

B. Port/Create a few application to show and use the benefits of the OS and whatever it brings to the table.

C. Implement an equivalent of WSL2 which should give broad compatibility at the relatively low cost of keeping up with a Linux kernel, given the kernel's rather good backward compat record we wouldn't have to be very prompt with porting new versions.

At which point users would have a good reason to use the OS (B.) and no practical shortage of apps (C.) even if the experience with these apps would be somewhat lacking. One can then concentrate on improving the OS or on porting the most used applications. This acts like a Cuckoo - in the beginning, there'll a mostly Linux userland with a few AltOS apps. Eventually one ports enough so that the system is really AltOS with a few Linux apps on the side.

I don't know how much work it was to do what waddlesplash has already done, but it seems like it's enough for GTK3, and almost enough for Xnest. If they get Xnest to work, then they can use that for anything that is doesn't use xlib, which sadly isn't a lot[1].

Maybe GTK4 would be easier to build a Haiku backend for (as was done for QT), but that doesn't help anybody run GTK3 applications on Haiku does it.

[1] xlib is kind of terrible IMHO; it adds mandatory abstractions that are superfluous at best and in many cases (again IMHO) harmfully force synchronicity on the asynchronous protocol that is X11. Lots of xlib based programs wait for a response when they have other things they could be doing, and the result is poor experiences blamed on X, when xlib's fault. sigh

> xlib is kind of terrible IMHO [...] force synchronicity on the asynchronous protocol that is X11. Lots of xlib based programs wait for a response [...]

Was fixing that the intent behind libxcb?

(honest question, my understanding of X stuff is limited, I just feel to recall Awesome depends on libxcb instead of xlib and read some vague blurb about it back then)

> Was fixing that the intent behind libxcb?

Yes.

Like I noted in another comment, XCB is basically just a loose wrapper around the X11 protocol. It is indeed a lot more verbose than Xlib, and modern Xlib is actually written on top of XCB, so you can use the two in the same application on the same X11 context if you really need to for some reason.

> If all you care about is GTK, then you could have just done a native port of GTK. That would actually be a lot easier than trying to reimplement every X client library.

LOL! Do you really think I woke up one day and said "hmm, I need a GTK port. I know, I'll reimplement Xlib!" without even taking a look at the GTK codebase? Of course I looked at the GTK codebase, and so did a number of other Haiku developers, and we came to the conclusion that writing a native Haiku backend would be a ridiculously unpleasant task (as one other Haiku developer pointed out, GDK backends are typically > 1 MB of source code, and in a dialect of C that I am, uh, if I put it politely, not a fan of.)

It turns out that writing an Xlib compatibility layer was, in the final analysis, a far more expedient and efficient way to port GTK. If I went the long way around, I would have had to write probably over 100k lines of code across GTK, Cairo, and who knows how many other projects (I stopped counting how many dependent libraries I built in the first stages of this attempt.)

Meanwhile, Xlibe does the job with only ~7000 SLOC. That's compared to ~23000 SLOC in the "gdk/x11" folder alone, not to mention all the other X11-related files scattered around the GDK and GTK source trees, or all the ifdefs throughout the codebase, and all the other projects that GTK depends on which interact with the X11 layers in GDK. There is a reason it took literal years to get GTK on Wayland natively...

> With GTK4 it should be even easier because the backends were simplified and a lot of the weird X idiosyncrasies were removed

I find that hard to believe given how deeply some of those X idiosyncrasies are embedded.

But anyway, supposing you're correct: Why should I care about GTK4? Even GIMP of all projects, the (more or less) flagship GTK application and one of the primary reasons a GTK port is generally desirable, has decided that for now, a GTK4 port is "out of scope" (no, really [1]) at least for the immediate future. So even if a GTK4 port was more technically feasible, why exactly would I care if none of the software I want runs on GTK4?

Oh, and let's not forget: with an Xlib compatibility layer, I can also get GTK2 and then whatever applications are still stuck on that; plus plenty of other X11-only things which otherwise I wouldn't be able to get.

[1]: https://gitlab.gnome.org/GNOME/gimp/-/issues/6440

> you might actually have a chance to get hardware acceleration to work without having to go through the pain of dealing with GLX

I didn't yet try to implement GLX but it looks more than feasible. Sure, it's not the most pleasant of APIs to deal with, but implementing it doesn't look so bad.

> The problem with Xlib is that's is not just the one library, it's actually about 30 libraries.

If you mean libraries like Xt, Xaw, Xmu, Xpm, etc., those just run on top of Xlib itself, they don't interact with the X11 protocol directly, and so I can and did build and use those unmodified directly from X.org. For all they know they are running with a strange X11 server that provides an odd but completely valid featureset. Xeyes, which you can see a screenshot of in the README, uses Xt and Xaw, so clearly those work already in this environment.

> Nobody has gotten that far, and if you do get that far

I think I have gotten "that far." I can run Xt/Xaw applications (albeit with a few font rendering glitches, though I just didn't bother to patch that), I can run GTK, and sure, I didn't implement every single drawing mode or every quirky feature, but I don't really care about running Motif, so why does that matter so much?

> So the goal here can't really be reached in any practical terms, unless you just want to stop here.

How about this: You tell me some X11 application you think I can't run under this, and we'll just see if you're wrong (or why nobody cares.)

>LOL! Do you really think I woke up one day and said "hmm, I need a GTK port. I know, I'll reimplement Xlib!" without even taking a look at the GTK codebase?

No. I would also greatly suggest against making these wrong assumptions. But from this comment it actually does sound like you didn't look at the GTK4 codebase.

>It turns out that writing an Xlib compatibility layer was, in the final analysis, a far more expedient and efficient way to port GTK.

This analysis is extremely incorrect and wrong. I'll get more into the details.

>If I went the long way around, I would have had to write probably over 100k lines of code across GTK, Cairo, and who knows how many other projects (I stopped counting how many dependent libraries I built in the first stages of this attempt.

Implementing the full X11 protocol with all the extensions and all the quirks in the server is much, more more than 100k lines. Also this entire sentence is extremely incorrect and I'm very confused as to what you were trying to do, you don't have to do anything in Cairo or Pango because those support client side rendering and have supported that for a really long time now. I assume you were basing this analysis off an ancient Haiku port or something that was trying to do things the old way? That stuff is obsolete.

>Meanwhile, Xlibe does the job with only ~7000 SLOC. That's compared to ~23000 SLOC in the "gdk/x11" folder alone

This comparison is complete nonsense. Please don't sink yourself to the level of doing this. The GDK X11 backend includes support for a bunch of extensions that you didn't implement. So no, it's not "doing the job" at all. IIRC if you look at the GTK4 backends for the more modern windowing systems (Wayland, macos) those have both shrunk to around 13000 SLOC. A native Haiku backend would probably be even shorter if you dropped all the optional features and only implemented the bare minimum, I bet you could do it in... around 7000 SLOC. There is nothing magical about any particular approach to this, it's all the same code you have to write in either case. Draw a box here, render some text here, translate these input coordinates, etc. You can't avoid this by implementing Xlib or doing any other approach.

>and all the other projects that GTK depends on which interact with the X11 layers in GDK.

Well actually those will still break with your approach if they link against the other libraries or depend on some X server functionality.

>There is a reason it took literal years to get GTK on Wayland natively...

Actually it took years because the protocol was still being worked on while the backend was being developed.

>I find that hard to believe given how deeply some of those X idiosyncrasies are embedded.

Well it's true, they were removed in GTK4, the entire rendering pipeline was rewritten.

>I didn't yet try to implement GLX but it looks more than feasible.

You could get it to work but it's not going to be feasible without also hacking the mesa implementation or re-implementing several parts of mesa yourself along with the GLX extensions. IMO this is not a good way to spend your time, please don't waste your donors' money on this. If you have a working EGL backend already you can avoid all that and just use the EGL support that GTK has already.

>But anyway, supposing you're correct: Why should I care about GTK4?

I don't know, you tell me. When you say "GTK" to me that means all versions of GTK that are currently in use by applications. If all you care about is GIMP then I would still say a better approach is to do a native backend because then you can also just skip everything that isn't used in GIMP. It's the same approach really.

>Oh, and let's not forget: with an Xlib compatibility layer, I can also get plenty of other X11-only things which otherwise I wouldn't be able to get.

Well no, this is also wrong, to get the full X compatibility (including network support) you will still need to port the full X server. And if someone ever wants that, basically all the work you did is going to be a waste and will have to be thrown out because then you're right back where you started with needing an X server. This is not a new revelation, Linux distros have been through all these conversations before, several times over the last 30 years in fact. I'm not telling you this because I'm trying to give you a hard time, I just wish other OSes would stop repeating the bad mistakes made by Linux.

>If you mean libraries like Xt, Xaw, Xmu, Xpm, etc., those just run on top of Xlib itself

No this would be the other libraries that implement protocol extensions.

>How about this: You tell me some X11 application you think I can't run under this, and we'll just see if you're wrong (or why nobody cares.)

You're being unnecessarily combative about this, please stop. I actually wish I was wrong about this, and it was as easy as you're suggesting, but experience says it's not, and also the problems that you've stated with your approach aren't helping either. But anyway, according to what you said, right now it's any application that requires XInput or XInput2 or requires 3d acceleration or wants to get run over the network. Or even just stuff like GIMP will have degraded functionality, for example you can't use drawing tablets with that right now. I would really advise against making these kind of comments if you're trying to represent a project or get more funding.

> I would really advise against making these kind of comments if you're trying to represent a project or get more funding.

Look, a little snark never hurt anybody.

The fact is that you are being extremely pessimistic about whether this will work at all, when in fact it is already being used "in production," and most of the other things you are saying are not possible or very difficult I have already investigated and made similar assessments to the ones I made before I started this.

So all evidence points to you being largely wrong, and your supreme confidence that you are correct despite all the evidence to the contrary makes it quite attractive to have a bit of fun while trying to demonstrate where you are incorrect.

That's nothing new in open source, or even software development in general, where much worse like actual insults and derogatory comments are much more an unfortunate norm, and toxicity is often the rule of the day. Plenty of stuff that goes on (or at least used to) on the Linux kernel mailing list would get you warned and then banned from Haiku's discussion channels. But at the same time, there is a place for criticism, even strong criticism, in any healthy discourse.

> But from this comment it actually does sound like you didn't look at the GTK4 codebase.

I gave it a cursory glance, but indeed I didn't look at it much in depth, because all the big GTK-based applications don't yet run on it, so it was a moot point.

> Implementing the full X11 protocol with all the extensions and all the quirks in the server is much, more more than 100k lines.

Sure, but I have already stated, multiple times, that we neither need nor want to implement all the extensions and quirks.

> you don't have to do anything in Cairo or Pango because those support client side rendering and have supported that for a really long time now.

GDK's backends presume Cairo has support for native "surfaces"; e.g. the GDK X11 backend uses Cairo's Xlib interfacing layer to draw stuff to the screen. I didn't look too closely, but the other backends seem to make similar usages on Windows and macOS. So I would have probably had to either revive the BeOS interfacing code in Cairo, or would have had to do lots of software copies in a Haiku backend of GDK in lieu of a Cairo interface.

Pango indeed doesn't deal with native "surfaces", so that wouldn't be a problem.

> I assume you were basing this analysis off an ancient Haiku port or something that was trying to do things the old way?

Nope. There may have been some work on GTK 1 for BeOS or something like that, I can't recall, but I didn't go looking for it, much less analyze it.

> The GDK X11 backend includes support for a bunch of extensions that you didn't implement.

For now. Some of those extensions (like XSync and XInput2) are probably worth implementing. This also ignores all the other X11-related code scattered throughout the GTK/GDK trees, anyway.

> A native Haiku backend would probably be even shorter if you dropped all the optional features and only implemented the bare minimum

Okay, but that's again GTK4, and totally ignores GTK2 and GTK3 which quite a lot of applications still use, and the entire point of a GTK port is to run GTK applications. Not much use in porting a version of GTK that very little supports.

> You can't avoid this by implementing Xlib or doing any other approach.

No, but I can decide that the GTK codebase is so far outside the realm of my and the other Haiku developer's programming tastes that we'd quite literally rather write an X11 implementation than deal with it. Even the minimal time I had to spend with the GTK code in a debugger while fixing incompatibilities in Xlibe was more than enough, thanks.

Furthermore, GTK is entirely C while Haiku's APIs are C++. Is GTK ever going to accept a C++ backend upstream? Almost certainly not. So if I am going to have to maintain this totally outside GTK, forever, why not do it in a way that's much more resistant to GTK changes?

And, again, there's more than just GTK here. Someone took Xlibe and apparently got some audio plugins from Linux (LV2, I think) to work with minimal code changes.

> Well actually those will still break with your approach if they link against the other libraries or depend on some X server functionality.

Again: which other libraries? You mean like Xinerama or something? If something actually needs Xinerama, or any of those other libraries, then we can implement those, too. (So far, those have turned out to be entirely optional, I haven't yet tried to build something that couldn't just disable those features.)

> Well it's true, they were removed in GTK4, the entire rendering pipeline was rewritten.

Okay. What about input events? Does GTK4 still have keysym values that map 1:1 to X11 keysyms, or did they break with that?

> Actually it took years because the protocol was still being worked on while the backend was being developed.

It's my recollection that Qt had a working Wayland backend long before GTK did. Am I incorrect?

> You could get it to work but it's not going to be feasible without also hacking the mesa implementation

I don't intend to hack the Mesa implementation of GLX, I'd just implement GLX on top of the Haiku OpenGL APIs directly (which use Mesa internally but that's an implementation detail.)

> IMO this is not a good way to spend your time, please don't waste your donors' money on this.

GLX is unimportant enough that I don't intend to spend time on it at present anyway. When/if there's a demand for it, I'll take a stab at it. But I've worked with GLX before, and I don't think it'll be nearly as much a challenge as you seem to think it will be.

> If all you care about is GIMP then I would still say a better approach is to do a native backend because then you can also just skip everything that isn't used in GIMP.

There's also Inkscape, wxGTK, and plenty of other GTK applications besides GIMP. Not to mention X11-only applications like AzPainter, etc. So it's not "just GIMP," but if whatever GTK version I'm porting, GIMP doesn't support, that's a good argument I picked the wrong GTK version.

> Well no, this is also wrong, to get the full X compatibility (including network support) you will still need to port the full X server.

Yeah, but I wasn't talking about that in the section you replied to, I was talking about things like Tk or AzPainter or the like which have X11/Xlib backends, and so by implementing this we also get those, not just GTK.

> And if someone ever wants that, basically all the work you did is going to be a waste and will have to be thrown out because then you're right back where you started with needing an X server.

Not if you port the X server using this, via Xnest! Which, like I mentioned above, I did get to compile and attempt to start but it crashed, and I decided I wasn't going to spend more than five minutes poking at it in a debugger because I didn't care enough about having X11 forwarding. If someone else wants to, then I imagine it will be much easier to patch the X server and Xlibe than writing a totally new X11 server backend for Haiku.

> This is not a new revelation, Linux distros have been through all these conversations before, several times over the last 30 years in fact.

I am quite aware, I've read some (or even many) of them.

> I just wish other OSes would stop repeating the bad mistakes made by Linux.

Hear, hear!

But this isn't one of those. As you pointed out yourself, nobody else did this before successfully; and it seems we've now already been more successful than Linux. Isn't that evidence that, just maybe, we aren't repeating a mistake here?

> No this would be the other libraries that implement protocol extensions.

As noted above, either most applications treat those as optional, or we can just implement them, too (Xft for instance would be pretty easy.)

> You're being unnecessarily combative about this, please stop.

I'm not sure why I couldn't say the same about you.

My offer still stands: tell me the name of some X11 application you think I can't run under this (obviously stuff like "window managers" don't count), and I'll make a pass at it and we'll see what happens.

> I actually wish I was wrong about this, and it was as easy as you're suggesting, but experience says it's not

I didn't anywhere say it was "easy." I'm drawing on many years of experience with both the Haiku APIs, the X11/Xlib APIs, graphics programming and widget toolkits in general, and even with that there's a lot of good old trial and error.

> But anyway, according to what you said, right now it's any application that requires XInput or XInput2

Quite literally because I just didn't implement those yet. I read through the XInput2 headers briefly and while they look particularly unpleasant in some ways, nothing looks technically impossible or even difficult, just tedious to deal with.

Plus it's another thing we would only have to do once: implement XInput2, and now all GTK applications, AzPainter, and anything else that can support it now has drawing tablet support on Haiku.

I'm curious why this approach was chosen for GTK, whereas Qt was ported directly. Is there some detail about GTK that made the approach taken with Qt infeasible?

Yes: the GTK codebase is, uh, not fun to work with and write backends for. See my comment above (https://news.ycombinator.com/item?id=31192099) and also one of my replies on the Haiku forums (https://discuss.haiku-os.org/t/xlibe-an-xlib-x11-compatibili...) for more details.

7 kLOC is a lot smaller than I had guessed this must have been. Pretty cool!

I am not even sure about what quirks you mention from Xlib. That said, I do think that having this server-less is going to be challenging. Even if the author claims this would ease things like DnD, I am not entirely convinced...

Similar thing but for Win32 - http://libw11.sourceforge.net/

> serverless

So this is a compatibility library? Not just a server that runs without a root window?

> but this works sufficiently well that GTK3 is now in Haiku's default package repositories, with GIMP and Inkscape atop it to boot, using this

That is absolutely fantastic:)

> And for the fun of it, I did try to compile Xnest (the X.org server variant running on top of Xlib) on top of this; it crashed on startup in the keyboard handlers

I don't have a Haiku box handy to test - is Xephyr any easier/better? It seems to be the "modern" option.

> So this is a compatibility library? Not just a server that runs without a root window?

Correct!

> I don't have a Haiku box handy to test - is Xephyr any easier/better? It seems to be the "modern" option.

Both Xnest and Xephyr are both "backends" for the main X.org server, though at two different "levels" and with two different goals. The main technical difference is that Xnest runs on Xlib, and Xephyr runs on XCB -- which is a much lower-level library that more or less deals with the X11 protocol directly, as opposed to having a slightly abstracted view of things as Xlib provides. (Modern Xlib is itself implemented on top of XCB.) So, since Xlibe implements Xlib, Xephyr isn't an option.

Thanks for clarifying. I assume the reason you implemented at this level is because it was a better match for Haiku's graphical APIs? And if you don't have xcb, does that limit what applications can run with this library? I think I remember seeing programs that depended on XCB directly.

> Thanks for clarifying. I assume the reason you implemented at this level is because it was a better match for Haiku's graphical APIs?

It's possible to integrate with the rest of the system much more smoothly than a full server will allow. GTK apps, for instance, feel like second-class citizens on basically anything except maybe GNOME (but do they even feel "at home" there...?) so it just didn't seem worth the effort to write a full GTK port vs. writing this, instead.

I discussed more of the "why" on the forum posts introducing this last winter: https://discuss.haiku-os.org/t/xlibe-an-xlib-x11-compatibili...

> And if you don't have xcb, does that limit what applications can run with this library? I think I remember seeing programs that depended on XCB directly.

There are a few (I believe Qt on Linux now uses XCB instead of Xlib), but not enough that it really matters (we have a native Qt port on Haiku already, anyway, e.g.)

How hard would be to abstract the Hauku-specific bits into their own part? It'd be neat if it was possible to use this not only to target Haiku but also other window systems like Win32 and Cocoa.

Does something like Gtk1.2[0] (which AFAIK does more heavy use of xlib than modern GTK) work? How about Motif?

[0] http://runtimeterror.com/pages/badsector/gtk12/

> How hard would be to abstract the Hauku-specific bits into their own part?

Hard, as I mentioned to other commenters, Xlibe makes heavy and direct use of Haiku APIs everywhere to do just about everything. However...

> It'd be neat if it was possible to use this not only to target Haiku but also other window systems like Win32 and Cocoa.

Using Xlibe as a starting point to write an equivalent library for Win32, Cocoa, or even Wayland would probably save you a lot of time, effort, and hassle in that a lot of Xlib general architecture, some behavioral oddities, etc. are already worked out in Xlibe as a matter of course. I'd be happy to guide others through what that process might look like if there was interest.

> Does something like Gtk1.2[0] (which AFAIK does more heavy use of xlib than modern GTK) work?

I didn't try it, but modern GTK actually makes pretty heavy use of Xlib; and Xt/Xaw do as well, which also work, so I imagine it would; or if it didn't, improving Xlibe so that it did would not be too hard.

> How about Motif?

Motif relies heavily on strange and implicit behaviors of X11. I was advised by one of the Emacs developers that it was probably inadvisable to try and get it to run under this. However, I did build it and it at least starts and does display things, but there are drawing artifacts and events do not work quite right.

So, it's probably theoretically possible, but I don't think it's really worth spending the time and effort in doing so. It is probably a better use of time to refactor/rewrite whatever software still uses Motif as its UI toolkit to use something else instead...

While the Haiku stuff is scattered around the code as you say, at first glance it looks fairly easy to identify. For example, any function call starting with a capital ‘B’ seemed likely to be calling into Haiku while most of the ‘X’ stuff is Xlibe itself. The ‘B’ stuff that I saw was stuff like handles, points, and strokes ( lines ) that called into Haiku to do whatever Xlibe was trying to accomplish.

To my eye, it looked like it would be fairly straight-forward to port this over to any other graphical API that provided the same basic concepts ( windows, views, points, lines ).

You would have to modify code all over the place but the code looks easy to identify and easy to map to alternatives.

Am I looking at this too simplistically?

Yes, it's probably easy to identify the Haiku-specific bits and start replacing them, but some of the organization of e.g. the Drawables classes is kind of implicitly built around Haiku APIs, so it might be a little less trivial than it appears at first glance. Still, it's indeed possible, if someone wanted to try it.

Something that maybe interesting: running it under DOS. That would allow porting some apps to FreeDOS.

Very cool. I guess the drawing functions are mapped to Be drawing routines, which means probably using this on say Android would be a big effort.

They are. More than just the drawing routines, quite literally the entire thing directly invokes Be APIs. For instance XQueryPointer just calls directly into the Interface Kit to get the present mouse position, or XSetWMProperties directly modifies the Haiku-native window state, title, etc.

However, that being said, if you wanted to do something similar or equivalent on some other platform, you could probably copy quite a lot of the structure, and then delete the actual implementation and replace it with your own. I restructured things a bunch in the early phases of working on this, and there's a lot of boilerplate code (e.g. macros.c, large parts of the event dispatcher) which you could reuse verbatim; though it's mixed in with the non-boilerplate code.

There were some other projects I ran into that looked like they were more generic, e.g. https://github.com/Abestanis/SDL2X11Emulation (also check the forks of that -- though I'm one of them, I experimented with it a bit before starting on this thing; but it's very incomplete and SDL2 really isn't designed for this), but none anywhere near as advanced as this is now.

I glanced at the code earlier and my first thought was fact it could be adapted to provide Xlib for SerenityOS.

I have nothing to do with Serenity and have never even used C++ but your code was easy to read and so the possibility just jumped out at me.

This is neat.

I thought one time of bringing X apps to Windows in a source-compatible way by embedding an X server similar to Kdrive in a library that communicated with the application via a ring buffer or similar. It's cool to see similar work actually being done.

Interesting! I love stuff like this (compatibility layers like wine, darwin).

I think a better title might include mentioning what Haiku is, though 15 seconds of googling worked just as well. :)

Could this awesome work be used to getting X11 running in the browser?

No.

As I noted to another commenter, this makes heavy and direct use of the Haiku APIs to do basically everything.

Haiku actually does have a full draw-call-forwarding remote desktop system, with an HTML5 client to boot, and you could run this through that, but it's kind of "experimental". For straight X11, using something like Xpra with a HTML5 client is probably what you want.

I do believe this page has an X server in the browser, or at least it strongly appears to: https://magcius.github.io/xplain/article/x-basics.html

I not even done reading the page and fully understanding this, but my mind is blown.

> That's because this isn't just any old stipple image, that stipple is actually the background of a full X server session running in your browser using HTML5 canvas.

Thank you!

You want X11 in the browser? Mind if I ask why, what you would use it for, etc?

(I ask because, as a side project I've been working on something that lets me have a more responsive laptop by offloading resource intensive graphical programs to another machine, and streaming the UI back to me. I guess it could be made to run in a browser.)

Yeah, I have been playing with RDP/remote desktop solutions in Docker for the same need. Another option is using something like the Phoenix framework in Elixir.

https://www.phoenixframework.org/

What techniques are you using, is remote desktop or a new desktop application framework?

Might be useful for porting some classic X programs to Web.

Ideally apps should be ported using Haiku's native GUI.

Reply on news.ycombinator.com