Learn Wayland by writing a GUI from scratch | Hacker News Reader

Learn Wayland by writing a GUI from scratch

gaultier.github.io

189 pointsmmphosis49 comments

See also: Learn x86-64 assembly by writing a GUI from scratch https://gaultier.github.io/blog/x11_x64.html

Discussion: https://news.ycombinator.com/item?id=36153237 (638 points | 4 months ago | 146 comments)

I am learning x86-64 asm with it. It's great !

Thanks for posting this! I missed it when it was originally posted and I’ve been looking for something like this.

From that link I also learned about lobste.rs which is also new (and interesting!) to me. Now I need to figure out a way to get an invite…

To the kind soul who sent me an invite: thank you!

Can someone invite me too? Ive wanted to post there for some time :-)

Check your email.

Nice. I'm workjng through the x86-64 asm one right now.

Could a kind soul please send me an invite to lobste.rs as well?

https://wayland-book.com/

This book is nice if you want to:

- Understand Wayland itself

- Write applications without being dependent on GUI frameworks

- Write a compositor yourself

It's author is Drew DeVault.

The only PDF version of the book appears to be at https://github.com/rcalixte/wayland-book for now.

Are there plans to do the same on the original site? The print button didn't work for me.

No idea about that :)

There's so many ideas and separate things/topics in this: a protocol between the Wayland server and your client, an RPC mechanism, a protocol format serialisation, synchronization (lock), an async mechanism, an object registry, shared memory, domain sockets, event loop.

Now I haven't written any non-CLI GUI Linux applications in C. I once played with the Qt IDE. I feel it is pretty difficult stuff!

I think dbus is also really interesting too.

This causes me to think of the success of Nodejs and how it abstracts async and makes it approachable to anyone. Rather than pointers in C you have JavaScript object literals.

The most complicated GUIs I created were with Swing in Java and JavaScript.

Can someone more knowledgeable tell me what is GPU accelerated with Wayland? Is there a bitblit operation uses by compositor? And how would you write a high performance GUI that is maintainable. I'm aware Blink/Chromium uses Skia which I presume is accelerated.

In my JavaScript visualisation experiments with canvases - the page has started to slow down because I'm doing stuff on the main thread.

See this example - scroll down and you'll see a process visualiser and scroll down further and there's a graph. I'm doing parsing on the main thread https://replit.com/@Chronological/Processes3#index.html

I feel this would be so much harder to build in C.

I am kind of following the trends in GUI rendering tech - I am curious how you support rendering lists of hundreds of thousands of items without expensive update costs. The perennial react performance problem.

I'm following the approaches taken by Tauri and Rust desktop app designs. Looking for a silver bullet.

Wayland is not a GUI toolkit

It's protocols which handle the communication between GUI toolkits your desktop manager (e.g. make new windows, resize windows, scaling, etc.) and how to connect to the GPU driver.

I.e. wayland isn't GUI technology it's a level below it.

So on systems using Wayland every GUI toolkit, including Skia, uses Wayland internally.

Also Wayland is _only_ protocols, so questions like "is window composition GPU accelerated" are implementation details of the specific compositor (through yes in general they are).

This is also on of the differences with X. Another one is X had a bunch of features sprinkled in which belong to the space of GUI toolkits (most of which had been abandoned by most software long before Wayland become relevant). Wayland is more cleanly designed wrt. to this aspects.

Going further, protocols of Wayland are also notably about passing filehandles to GPU memory (DMA-BUFs) to the compositor. The client allocates the memory, the client draws to it, however it wants, and it uses the Wayland protocols to pass reference to what's it's drawing to the compositor. https://wayland-book.com/surfaces/dmabuf.html

In contast, X was born before os had anything like dma-buf. X has all kinds of tools for drawing.

There's a lot griping about how Wayland isn't a monolithic server, how different compositors have to reimplement base capabilities and how this could lead to incompatibilities or different capability sets. Personally I think the fear uncertainty & doubt is overblow (open progress and possibility trumps conservative consistency), but I think what's technically interesting about how Wayland compositors work is that the Wayland design leaves so much more to the OS & client than X did. It makes it much more feasible to make a new compositor, since really Wayland is just a buffer manager, for os buffers, allocated by clients. This reinforces, to me, your proposition that Wayland is much more clean, by having to invent so very much less than X had to invent (and it truly had to, there we're not good os level constructs to use, the time).

Thank you to both you and GP, you both gave me context that helped me understand Wayland’s role a lot better.

>There's a lot griping about how Wayland isn't a monolithic server

Implementation detail.

>how different compositors have to reimplement base capabilities

Implementation detail

If people wanted they could make a monolithic server. They could make window management happen by a different process that could be swapped out.

The problem there is that the Wayland protocol family is not tolerant of dropping events or buffering them indefinitely, so there is no benefit to an X server-like architecture where you can transparently restart or replace the compositor. You're welcome to try, of course, but there are an incredible number of scenarios that break clients; one off the top of my head is if the mouse cursor creates a client tooltip by hovering over some GUI element, then leaves that region while there is no compositor to handle the mouse movement events. You're going to end up in a world where you're synthesizing fake events to "reset" the state of clients, and this is dragons-at-the-edge-of-the-map territory.

I wish Wayland was architected around a state-synchronization model instead of being event-driven. It'd be extremely difficult to port client toolkits and would ultimately fail to attract adoption, of course, but you'd end up with a more resilient system overall, and one that could be a bit more implementation-agnostic.

The problem is we have to use Wayland now, but the ecosystem isn't quite at X11 levels. Most documentation and tutorials are X11.

Wayland will probably be amazing like many of other technologies I've ever disliked in the past (Some of them still seem to suck though!) but right now it's still effectively brand new on the general user scene.

dma_bufs can’t cross machine boundaries, so Wayland not being monolithic requires deploying another service directly onto the user’s desktop machine and having it draw everything on behalf of all the GPU-less app servers. In practice the web browser strangely takes this role, with Javascript or Wasm as both window manager and rendering API, but in this world I’m not sure why the compositor still needs to be there.

> This is also on of the differences with X.

X is a protocol too. Xorg isn't, it's an implementation. But there can be (and have been) others.

Never programmed with Wayland (or X) before but this doesn't look so terrible. Let's say I want to do something only slightly more complicated than this article, e.g. have a few static rectangles on the screen showing images and text, without any window decorations, like for a status bar or a clock or something. If I don't want to roll my own protocol-handling code as in this article, what libraries are good? In say Rust or Go or something? Every time I've looked into Gtk or Qt they seemed way too complicated, doing far more than I want and introducing too many abstractions and dependencies. Conceptually all I want to do is:

    window = make_stupid_simple_undecorated_window(x, y, w, h);
    paint_window(window, some_image_buffer);
    block_forever();

I've toyed around with Smithay Client Toolkit [1] for Rust and I think it offers just the right abstraction for writing simple Wayland clients.

[1] - https://github.com/Smithay/client-toolkit

Unless your "application" is the only thing in the screen, you need some protocol. At a minimum, repainting after your windows has been obscured by others, reacting to commands like closing the window or the session, shutting down, etc.

The main difference from CLI is that GUI works reacting to messages, even for (re)painting. Once you get that, the rest falls into place. This is usually implemented with object-oriented libraries, because it's the kind of application where OOP shines. But it's not mandatory.

Oh, and your app should also respond to move, change workspace, minimize... messages.

This was fun to read but also .. I'm so happy I don't have to touch C anymore.

There's an increasingly small list of reasons to build a GUI outside of the browser. Personally, I don't think I ever will have this need again.

Way too many chances for buffer overflows.

It is a nice approach to learn how things work, but not a recommended approach for building anything useful.

WAYLAND_DISPLAY lets you do a buffer overflow

The C code in the article is not great, but I don't think the author was trying to write a robust, production-ready Wayland client either.

A couple examples of fragile code, so my comment is a little constructive:

- I truly dislike the cstring_len macro, which sounds like strlen yet its semantics are completely different.

- The wayland_display_connect function returns EINVAL or an fd. Both are positive integers. It should instead return -EINVAL.

- The functions benefit from more comments, because they do a lot and are not factored to smaller, more composable units.

- wayland_handle_message should be refactored so it's a simple switch that operates on constants, and ideally delegates to other functions.

- Probably outside of the scope of a simple article, but all the buffer boilerplate should be simplified by using a buffer structure that holds pointer, cursor and capacity, to make the code much more readable.

And EINVAL is a perfectly valid fd.

> wayland_handle_message should be refactored so it's a simple switch that operates on constants, and ideally delegates to other functions.

Is that actually possible? Opcodes by themselves are not unique, and the object IDs are right now assigned at runtime (I can’t tell if that’s actually necessary).

Agree. Not sure about roundup_4() either.

> The wayland_display_connect function returns EINVAL or an fd. Both are positive integers. It should instead return -EINVAL.

Or maybe return -1 and set errno? I haven't seen many C programs that return negative errnos (excepting the linux kernel which uses negative errnos internally).

Why continue with the mistakes of the C standard and go through a thread-local global variable instead of just a return value? Convention is often not a good justification.

Not being production ready isn't an excuse when you are writing code that is intended to be copied by others.

I disagree. Didactic code doesn't have to be complete, just clear enough to teach. Nor the teacher is not responsible for lazy students.

It's lazy and quite ignorant to criticize teaching material because it doesn't cover all possible corner cases, and because it doesn't account for a reader that just wants to copy the code without understanding it.

There is a difference between not handling a corner case and having a security vulnerability. People are lazy and they will copy and paste without thinking.

Whose fault would that be? Geez, now blog authors need to have disclaimers, like reminding people that, no, you shouldn't eat Tide pods.

The topic is Wayland, not C code hardening techniques for internet facing services. Quit being an obnoxious pedant.

Oh, there are many exploits in this example code which could be demonstrated.

But this is not production code, and shouldn't be read this way.

In my opinion the writer quite clearly establishes that this code and methodology is only intended to be illustrative of the broader subject, which is: how Wayland works.

And inasmuch as it is illustrative, the very many exploits it shows are explicitly available, is a root note of its value. This author knows whats up.

The use of assert instead of runtime conditions for bounds checking doesn't seem like a good idea.

That's a very normal use if asserts - no point in a running but broken program if important programmer assumptions failed, and a coredump is more useful for debugging than an error message.

But C assert.h checks can be disabled by setting NDEBUG and sometimes are for production builds. It’s not meant for bounds checking or anything that required for correctness, but for asserting invariants, hence the name.

It's a blog post to teach about Wayland, not a production app. If one creates an app by copy-pasting this code, blame them, not the author.

Yes you can actively disable it. You can also pass "-gcflags=-B" to Go to disable bounds checking, and Rust disables runtime overflow checks in release mode by default. Doesn't make the use any less valid.

In-bounds accesses should be an invariant. A program that is capable of even attempting an out-of-bounds access is probably incorrect. With that said, I would not be confident enough to allow bounds checks to be disabled. (Rust is my favorite.)

Consider this - you may not be able to hit your assumption with all possible inputs in debug/during development, or your function may accept user-provided input, you then release your software, and then somebody gets your code executed with a set of inputs that don't pass the assumption, now since it's an assert that's been effectively turned into a no-op in release, you're reading past the end of the buffer.

asserts are not disabled in release builds unless you explicitly go out of your way to do so.

Various build tool chains disable them implicitly for release builds.

You should probably assert

#ifdef NDEBUG #error #endif

In your source code if your code assumes that assertions will never be disabled.

Honest question : what's the difference between the two?

I assume assert also checks conditions at runtime?

Not in release builds

Which can catch the unwary out, for instance if you do something like this it won't decrement index in your release build:

    assert(--index >= 0);

Reply on news.ycombinator.com