Writing to the Framebuffer (2018) | Hacker News Reader

Writing to the Framebuffer (2018)

seenaburns.com

205 pointsp4bl070 comments

My understanding was that fbdev is deprecated (but I was unable to find good info on this) and that one should use dumb buffers (https://manpages.debian.org/testing/libdrm-dev/drm-memory.7....) for this simple stuff / somewhat portable software rendering. At least Qt allows to do this for embedded devices without hardware rendering.

Embedded developers still need fbdev. There are lots of display-enabled SoCs that don't have or need a GPU.

But even then, not every display-enabled SoC is well modeled as a dumb framebuffer. The Raspberry Pi itself provides hardware overlays, and it's quite problematic to expose them under that kind of model. The DRM infrastructure seems to be a lot more general and flexible.

I wouldn't recommend an RPi for product development. There are better SoCs, like i.MX, with documented graphics pipelines and graphic/video/camera overlays that are easy to use.

Incidentally, i.MX6 is an example of using the Linux fbdev with hardware acceleration.

Yes. fbdev is kept alive for compatibility (and fbcon) reasons, but it's a really outdated API and drivers have to jump through hoops to support it. Please use dumb buffers when necessary. See a simple example here: https://github.com/magcius/drmdemo

Does the Nvidia proprietary kernel module provide a DRI interface?

Yes, since a few years ago. https://download.nvidia.com/XFree86/Linux-x86_64/396.51/READ...

How can I write a simple device driver using dumb buffers? I have searched far and wide but I can't find anything. I'm forced to write a framebuffer driver. Once you dip into DRI shit get's insanely complex quick.

For the simplest case, where you have contiguous memory for your framebuffer storage and no MMIO, try the CMA helpers: https://github.com/torvalds/linux/blob/master/drivers/gpu/dr...

I wrote a basic device driver for a platform I worked on. It's since been upstreamed and gotten more eyeballs, but it's still relatively simple. https://github.com/torvalds/linux/blob/master/drivers/gpu/dr...

This is something I exploit to do an early splashscreen to HDMI on the i.MX6. The kernel splashscreen is supported with LVDS but not HDMI on the kernel that ships for that SOC so as a workaround I take a bitmap representation in the color depth of the framebuffer and cat it into /dev/fb0 while in the initial ramdisk.

This is a similar tutorial in C.

http://betteros.org/tut/graphics1.php#fbdev

For a few months I've been kicking around the idea of making a mario clone using this. You boot the machine and you just get the game.

EDIT: for 2D graphics, you can blit sprites in with the CPU. GPUs are mainly useful for 3D graphics, physics and raytracing. Also, I made a mario clone in Unity and Unity felt like overkill.

> GPUs are mainly useful for 3D graphics, physics and raytracing.

GPU's are quite usable as sprite engines. A Wayland compositor is essentially blitting client-controlled sprites as "surfaces", and the final render is pixel perfect (other than perhaps during animations). There's no reason whatsoever to use software rendering if hardware acceleration is available.

> There's no reason whatsoever to use software rendering if hardware acceleration is available.

The really obvious one is that you might like to have a program that will work with or without hardware acceleration present.

>Another answer I found said no modern operating system will let you access the framebuffer directly. BUT LUCKILY THAT’S WRONG.

I don't know why this works for OP, but my understanding is that the original assertion is true. Writing/Reading directly from fb0 does not work on my machine.

Did you try what OP did to fix this for himself – i.e.

> sudo adduser seena video

No, it's not that I'm having a permission issue - /dev/fb0 simply returns all 0 on my machine.

Do you have any other framebuffer devices or multiple GPUs? Or proprietary video drivers?

Note: I was part of a small group that worked on a framebuffer utility library around 2002ish, but haven't used it much since apart from configuring embedded devices.

I'm getting the following:

  $ sudo hexdump -C /dev/fb0                                                                                                                                                                                                                                                                                                                                                          
  00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  *
  0000e000  aa aa aa 00 aa aa aa 00  aa aa aa 00 aa aa aa 00  |................|
  *
  0000e020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  *
  0000f000  aa aa aa 00 aa aa aa 00  aa aa aa 00 aa aa aa 00  |................|
  *
  0000f020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  *
  00300000

`*` for repeating lines.

You can't do it from an X11 session, which uses DRM/KMS instead of fbdev, hence you will get the compatibility fbdev buffer, which will be all 0s. You need to be VT switched away (hit Ctrl+Alt+F3 or similar).

This is such a cool blog post, and has pulled the covers back on something I have been curious about for some time.

Makes me wonder why more people aren't abusing this. I can see the gold, silver and platinum level open source sponsors shoving some big logos down your throat by writing directly to the framebuffer during your next yarn add.

(I realize it is more nuanced than that, since it requires elevated priviliges to write to the device ... but hey how many sudo curl bash scripts have you seen out there in the wild? time to pwn the framebuffer)

I believe most modern distributions have the framebuffer driver disabled. Embedded is an exception.

You can't write to the framebuffer device while X11 is running because it will just get overwritten immediately. So that means you can only use it on a machine without a display server running. So... a web server maybe? Sure, but since most people use a web server by SSH'ing into it, a program that writes to framebuffer devices is going to be useless, unless you have a physical display device connected to the server and want to display something on it remotely.

So it's mostly a fun gimmick/curiosity. I wrote a realtime 3D rasterizer that renders to fbdev a while back. It was a fun project, but useless.

Works even in JSlinux. https://bellard.org/jslinux/vm.html?url=buildroot-x86-xwin.c...

Download old qemu. https://drive.google.com/file/d/1e6UJcTC9FfwO20xzvcasTToPoaT...

Load into jslinux. And execute the following commands in xterm.

gzip -dc qemu-0.9.1.tar.gz | tar xf - -C/

mkdir /dev/shm && mount -t tmpfs tmpfs /dev/shm

SDL_NOMOUSE=1 qemu -k en-us -m 4 pi.vfd

Necessary condition, Xorg must work with the fbdev driver.

You can login to a tty terminal and run your program that way. Ctrl + Alt + F1-F5 on linux.

> You can't write to the framebuffer device while X11 is running because it will just get overwritten immediately.

You can't write to fbdev because that's not what X11 is using to display buffers to the screen, it uses the newer kernel mode-setting (KMS) API. It's only when you switch away to a VT that the "KMS master" drops its session, the kernel switches back to fbcon, and you can see the "raw fbdev" buffer again, which is usually a fake framebuffer. If you then write to the fbdev framebuffer, you'll see your changes, but fbcon will come in and overwrite them with the kernel console, unless you ioctl(KDSETMODE, KD_GRAPHICS); to turn off fbcon from drawing.

One of the weird things is that writing to the framebuffer can be surprisingly slow. At least I've seen it in practice where a machine that can play a video just fine in X maxes out the CPU and drops frames when trying to write directly to the framebuffer. It seems like it should be the opposite and since it was just something I was trying out for fun I didn't dive too deeply into figuring out what was going on. It may have been something like X was using GPU accel or the video player was writing to the framebuffer in an inefficient way (one write call per pixel) or something.

That sounds a bit off to me.

> writing to the framebuffer in an inefficient way

Fun fact you can memory map /dev/fb0 and make changes to it directly.

Decoding video frames is much more expensive, than rendering them. The source of the slowdown could be software decoding the video frame. Where the X video player might have been using a specialized chip for this, i.e. GPU or Intel graphics, depending on the setup.

For most video cards and driver combinations, the fbdev interface is an emulated, legacy interface. DRI and X have the intended fast path to the device, and you have to jump through some hoops to get to an API and handle that behaves like mmap'd /dev/fbX.

Chances are you are running yarn in a virtual terminal (iterm, xterm, gterm, etc). The framebuffer device doesn't represent your terminal window, but your physical display output (sort of, nowadays the framebuffer itself is emulated).

If you do use the fb device for some reason you don't need to have the user give the screen parameters like size and bit depth. Once you know the name of the framebuffer device all of that information is available in /sys/class/graphics/fbX/

There's also FBIOGET_VSCREENINFO / FBIOPUT_VSCREENINFO through which you can set the current screen information. You can also do fake double buffering this way, by allocating a mode that's twice as tall, drawing to different parts of it, and changing the yoffset to present each "buffer".

I have so many questions about this stuff! In particular though - im running a single GPU passthrough setup on my machine. It works smoothly, but I need to shutdown the display manager (gdm/lightdm etc. depending on distro) before unbinding the graphics card from the host and assigning it to the guest.

This means when I get back to my host, i have to relogin and all my windows/apps are gone. There seems to be no way to save/restore the state of X11 (i know apps themselves wouldn't save state). Is there something fundamental about how display managers tie into the GPU that makes it impossible? Is it possible to configure X11 and display managers to use software rendering only?

In the article it says you can copy `/dev/fb0` to take a screenshot. I tried this, but I don't know what format that would be to be able to display it or how to convert it to a usable format. Any ideas?

The format used by the framebuffer can differ based on your hardware and your display settings. You need to use a program that can load raw bitmap data, and it will need you to tell it what format the source data is in (resolution, byte order, etc).

There's a good explanation of this on StackOverflow: https://stackoverflow.com/questions/1645181/taking-a-screen-...

It will be raw bytes, format depending on what mode you're in. Usually 32bpp. Expect RGBA, but it might be ABGR or anything else. But you'll be able to figure it out quickly enough. Older hardware or weird modes might be more or less (16/24 bpp modes from the past, or maybe 10/10/10/2 for HDR out? I've never looked into that), but as it will be uncompressed it won't be much work to figure out.

It's raw pixel data, but in general you can't just cat /dev/fb0: only part of the buffer might be be displayed on screen, and there might be colormap involved (even in 24bpp or 32bpp modes).

I recommend using fbcat for taking screenshots: https://github.com/jwilk/fbcat

Isn't the graphics pipeline a bit more sophisticated nowadays? I.e., you could render your framebuffer on the side of a rotating cube, let hardware render fonts, etc.

Sure it is, but sometimes it's fun to dive into lower level stuff. Program your own software rasterizer or GUI just to see how that kind of thing can be engineered. Plus, modern CPUs are so fast, you can actually do a lot, easily get interactive performances with just software rendering.

Also, I would personally say that I feel modern APIs are getting a bit too complex. There's a lot of surface area and platform-specific details that you wish were portable but aren't and you unfortunately have to worry about. The amount of boilerplate code you need just to do a Hello World on something like Vulkan seems daunting. It's kind of nice to be able to build your own framework that you can completely wrap your head around and tailor to your own needs in every way. Then, all you need to worry about in terms of interfacing is how to display pixels at the other end.

Writing directly to the frame buffer is effectively the “CPU rendering” or “software rendering” approach. Most modern GUIs use a compositor, which usually renders the contents of each window using the CPU to textures stored in VRAM. The GPU then does the actual rendering, sampling the textures drawn by the CPU (the windows) as it fills in the polygons, defined as sets of vertices also defined by the CPU.

The term "framebuffer" is a bit overloaded here. In this case it means basically "monitor scan out buffer" and is therefore a very special, particular thing. In most other contexts 'framebuffer' usually just means your color output buffer, which is just a buffer of some kind, sitting in some RAM somewhere, with otherwise no special meaning.

When you setup a swap chain for GUI rendering, for example, you're just doing a glorified malloc call. It's not coupled to a hardware port. It's not really even all that special. And then like any other color buffer, you can use that color buffer in other ways, writing it to yet other color buffers.

All that said the display output pipeline itself still is more sophisticated than this. Not necessarily by all that much, but it is. Particularly in that there's not necessarily "one" buffer that drives the monitor. There's this idea that composition can happen "on the fly" during scan-out by dedicated hardware, which are done via multiple planes. How many planes can be used then depends on the hardware, but for example most Android phones have 4-8 hardware planes. Desktop hardware tends to have fewer such overlay planes (the power efficiency gains don't tend to matter much when you have infinite power), but modern Intel integrated GPUs I believe have 3 planes. This is particularly useful as one of those planes is for video. This means that when you're watching a video, if things are working properly, then the GPU can be basically powered off entirely.

A while back, a bunch of android devices (mainly cheap ones, low end Huaweis etc) had world readable frame buffers. Meant that an unprivileged app could read and process /dev/fb0 and snoop on the display. Same was sometimes true for /dev/input/eventN —- allowed touch screen to be read from and written to.

At some point Samsung flagship phones had world-writable /dev/mem

Direct I/O to the framebuffer may be very, very slow. It usually bypasses all caches and is treated as a synchronous device register access, which means the CPU waits for it to complete. And it may be one byte at a time. It's mostly a legacy mode for debug purposes, boot screens, and such.

Anybody knows why sudo did not work?

Redirect to a file is done by the shell, not by sudo process.

It even tells you that:

    > -bash: /dev/fb0: Permission denied
      ^^^^^

it only runs `cat` as root, not the write operation

This will only run `cat /dev/urandom` as root, while the redirection is run as a normal user:

sudo cat /dev/urandom > /dev/fb0

This should work just fine (with a /dev/null redirection so your terminal doesn't get garbled):

cat /dev/urandom | sudo tee /dev/fb0 > /dev/null

When I encounter this limitation and can not quite remember the correct incantation to solve it, I usually do something like this in frustration:

   echo "cat /dev/urandom > /dev/fb0" | sudo sh

I was quite proud of myself the first time I figured that out many years ago :)

Alternatively:

    sudo sh -c "cat /dev/urandom > /dev/fb0"

I noticed this a while ago. I've always assumed it's because the redirection is being performed by the current shell running as $USER and not the new process launched as root by sudo.

Permission problem for the redirection.

From https://stackoverflow.com/a/82278/5208540 , here's how to "redirect with sudo":

Use "| sudo tee /dev/fb0"

tee will also write to stdout, which can often mean that you get a bunch of binary gubbish splattered all over your terminal.

I recommend instead using "|sudo dd of=/dev/fb0", or, if you have moreutils installed, "|sudo sponge /dev/fb0"

But 'sponge' will put it all in RAM, so that's not ideal either. Lots of pitfalls here, wow.

Shouldn't `| sudo cat > /dev/fb0` work?

How do you select the video mode? (i.e. resolution, refreshrate, bits-per-pixel, etc.)

https://stackoverflow.com/questions/34904763/linux-framebuff...

And https://www.kernel.org/doc/html/v4.15/gpu/drm-kms.html

(Not parent) Thanks! I needed this a while back for a bodge. Ended up stashing it, but this will help a lot!

The kernel modesetting (KMS) API.

For another fun graphics project try getting Qt5 to compile using the EGL target on a basic Debian system without using X11 or Wayland/Weston.

Welcome to the era of MS-DOS A000:0000 direct video ram drawing frameworks. And all sorts of tricks of that time using Borland Pascal and Turbo C.

I just remembered Matrix 1 and Neo's framebuffer got hacked during the night when he got invited to the rave party.

I really think it's a shame the framebuffer console can't do proper bold, italics and underlines...

fbcon emulates the VT-100. The VT-100 has no support for text formatting, unfortunately. If a new standard was proposed and you could get acceptance, similar to some of the other newer protocols like displaying images in the terminal championed by iTerm, I'm sure Linux would accept a patch for it. I don't think it's a terrible idea.

That is after the fact rationalization and wrong.

The "new standard" has existed since 1976, which had boldface, underline, italics, faint, underline, reverse, concealed, strikethrough, and blinking; which gained things like overline by the 1980s; and which has been widely adopted for over 40 years.

And the Linux built-in terminal emulator is not emulating a VT100. The idea that terminal emulators emulate VT100s is generally wrong. Almost always they are doing what VT100s never did or could do, and conversely often not doing what DEC VTs did (such as supporting Tektronix, VT52, printers, windowing, multiple display pages, sixel, locators, ...). The Linux built-in terminal emulator has several significant differences from a VT10x, not the least of which is that it is not monochrome. The Linux built-in terminal emulator's closest relative is, rather, the SCO Console.

(Interestingly, SCO's manual relates that the SCO Console actually did have underline et al., when used on MDA hardware.)

The real problem was that boldface and italics quadruple the kernel font memory requirements, in a system that is still trying quite hard to limit itself to 512 glyphs. It's surmountable, but in many respects pointless. The headlined article is quite apposite, as there exist quite a range of terminal emulators that run in user space and realize using the frame buffer.

In user space one can be a lot more free with memory, loading multiple fonts simultaneously for example, and handling the whole of Unicode. And of course one can implement a whole bunch of the ECMA-48:1976 attributes and colours, including some of the more recent extensions such as the kitty underlining variants, AIXterm colours, XTerm colours, and ITU T.416 colours (where "more recent" for the latter three transalates to "from the 1990s").

> supporting Tektronix

That'd be awesome, but it'd need to be green, flash bright when drawing and reverse-flashing the screen on erase. ;-)

> The real problem was that boldface and italics quadruple the kernel font memory requirements,

In ancient times we OR'ed a character with itself shifting it one pixel horizontally. Italics were usually done with shifting the top part of the cell to the right and the bottom to the left. Some terminals could shift things by half a pixel, which made for great screen fonts (I did that on Apple IIs).

The same way overlines, underlines and strikethrough are trivial to generate without adding any glyphs to the font.

I know.

* https://github.com/jdebp/nosh/blob/79b1c0aab9834a09a59e15d47...

But nowadays people expect to provide actual font files for weight and slant changes, as boldface is not actually overprinting and italics are not oblique. This starts to matter at cell sizes bigger than 8 by 8, and it isn't an 8 by 8 world nowadays. You'll notice that my terminal emulator reads different font files for bold/faint/italic combinations if they are given. (On one test machine I am currently feeding it a combination of UbuntuMono-i and UbuntuMono-n, with unscii-16-mini and Unifont 7 as fallbacks.)

* https://github.com/jdebp/nosh/blob/79b1c0aab9834a09a59e15d47...

It is not alone. The FreeBSD kernel's newer built-in terminal emulator (vt) loads two fonts, one for normal (medium) weight and one for boldface.

* https://github.com/freebsd/freebsd/blob/5f17a06d9cc0a87e26fb...

You still can programatically generate those styles at larger grid sizes. OR and a single pixel shift will work at 8x8, but if you have enough pixels, you may need to OR more horizontal and vertical versions to get the effect.

I would prefer better, specifically created font styles (as the good terminals had), but I'd be totally happy with whatever gave me support for character styles. I swear that if I could figure out how to add them, I would.

That and smooth scrolling. ;-)

> I'm sure Linux would accept a patch for it.

I'd need some guidance (I have no idea of where to start), but I'd totally do it.

Mine can. (-: In fairness, so too can some of the others.

Reply on news.ycombinator.com