Hacker News Reader

Comment by ks2048 | Hacker News Reader

ks2048

I've only dabbled a bit in 3D graphics (OpenGL, THREE.js, swift SceneKit). When these 2D GUIs say "GPU-accelerated", does that mean they are doing the same thing you would do for 3D - build triangular meshes, materials, etc - and then just essentially point an orthographic camera at this "scene"? Or what kind of low-level GPU APIs (eg WebGPU) are used?

View on HN

> When these 2D GUIs say "GPU-accelerated", does that mean they are doing the same thing you would do for 3D

Yes. Nowadays all modern desktop interfaces—Aero/Metro/WinUI2/3 on Windows, Aqua/Cocoa on macOS, KDE, GNOME, XFCE, LXDE, and even some window managers on Linux—are 'GPU-accelerated'.

Every window is a quad of two triangles. There's no real vertex shading since it's all orthographic as you mentioned. The framebuffer for each window is exactly the x:y resolution for that window (macOS does some interesting 1:2 resizing here sometimes). The 'fragment shaders' is where the GUI toolkit comes in, writes to these buffers, and does any decorating where needed.

The final framebuffer is exactly the resolution of the entire monitor (again, macOS may do some weird 1:2 resizing).

The framebuffers of all windows on-screen are composed into this one. This is where things like transparency effects, the window and scroll controls, drop shadows, any 'rounding off' masks (used to great extent in macOS), and funky 'frosted glass'/'reflection' effects come in. This gives the effect of windows behind/in front of other windows. This is also when partially/fully off-screen windows are clipped/culled against the viewport frustum (not really a frustum but more a cuboid since it's not a perspective).

Once all this is done, you have a frame that's ready to be piped down the display cable into the display.

There are some other facets muddying the water like HDCP DRM protection for the entire framebuffer or some window framebuffers, variable-rate refresh, and so on. The former is how PrintScreen on Windows returns a black screen for some windows—that's HDCP in action.

Sorry just have to pop in and say that the DPI scaling makes the concept of "resolution of the window" much more complicated since you have some logical resolution and then the actual rendering resolution and two may not necessarily map 1:1 if DPI scaling is in effect.

Most of the drawing work of a GUI is drawing text, shape paths, and images. A GPU-accelerated UI layer draws these using GPU commands. It's not exactly the same as making an ortho 2D scene, but conceptually that's pretty much it. Often it means using crude geometry (such as a quad per glyph in a text renderer, or a "ribbon" of quads to cover a curve) and using a shader to draw the glyph or curve itself.

For a simple example, think of a rounded rect. Typically this would draw as a quad (a pair of triangles), and a shader would calculate the rounded corners.

There's also a lot of compositing and clipping that happens in a UI (e.g. a large widget inside a scrollbox) which is challenging to do on GPU as these get nested.

Modern 3D rendering APIs (like Metal, D3D12 and Vulkan and D3D11 to some extent) offer far more flexibility than the traditional triangles-and-materials approach. Rendering in Brisk leverages complex shaders that directly process drawing commands on GPU, avoiding the intermediate step of converting them into a list of triangles.

For text rendering, Brisk uses FreeType to render glyphs on the CPU and caches these glyphs in a GPU-accessible texture, which is reused for improved performance. This approach is common among GUI toolkits for handling graphics and fonts.

In addition to this, Brisk employs SDF (signed distance field) graphics wherever possible, which are entirely computed in shaders.

I think it just means the texture image for the widgets are loaded into gpu memory space instead of a traditional framebuffer that gets copied to the gpu.

I dabbled in 3D for a while too and was astonished how much 2D stuff there is for it.

Reply on news.ycombinator.com