Fedora 36: A brave new (DRM/KMS only) world
blog.dowhile0.org
blog.dowhile0.org
If so:
Dear video gods,
A fickle wizard told me I could fix studdering Chromium playback on HBO Max for my RPI400 by reciting the following incantation:
"remove the letter f from the line `dtoverlay=vc4-fkms-v3d`"
I did as instructed. But now the wizard's spell has caused my mouse pointer to become drowsy-- it lags when I drag it in the Chromium window.
Please help me!
Also, if you could explain to a mere mortal what the hell that setting is, I would be grateful.
Also if you could explain why the "Direct Rendering Display Compositor" for Chromium is inescapably Disabled no matter what flags I set, I'd be grateful. It seems like that affects the efficiency of subtitle display, but I'm just guessing here. (Using Raspbian Buster on the Rpi400.)
In your honor I offer this ASCII goat:
(_(
/_/'_____/)
" | |
|""""""|DRM is the direct rendering manager, it's mainly an interface for applications like X11 to talk to the GPU directly (ie 3D Acceleration and Video Decoding).
If KMS is causing things to be slow, it might be because the video driver is not working and the software framebuffer is being used. At higher resolutions that'll be slow.
If Chrome doesn't have the option to enable DRDC then the driver used likely doesn't support DRM or the window manager is not passing it properly.
But then how could using a software framebuffer on an underpowered thing like the RPI400 cause the video to play smoothly?
I would not call Raspberry Pi 400 underpowered.
The Cortex-A72 cores are fairly fast.
vc4-fkms-v3d is a module, vc4-kms-v3d is an entirely different module. These correspond to files somewhere on your device that describe the device and how to operate it.
The -f- version seems to do more video processing in the firmware rather than in the kernel. I can't tell you exactly what they do differently, but I'd assume that the -f- variant uses some kind of Broadcom magic and the non-f-version runs mostly in-kernel and therefore has different side effects when it comes to video processing.
Switching between these changes the way the kernel deals with graphics, so that's probably why you see the mouse being smoother in one version and hardware acceleration being unreliable in the other.
I can't tell you why Chromium is being weird. It could be a Wayland/X11 issue, it could be a driver support issue, it could be firmware issue, or maybe it's got something to do with sandboxing (Ubuntu and its Snap sandboxing have caused hardware access issues for me in the past). I have no idea what DRDC is even supposed to do, I'm guessing it's an optimisation in the Chrome rendering engine but who knows at this point.
To get the answers you seek, you'll have to dig into source code, I'm afraid. Perhaps if you reserve engineer the requirements for Chrome to enable the setting you'll find a hint at what you need to do to your rendering software to get it to support Chrome's code.
What does it mean to do video processing "in the kernel?" I assume that cannot mean software decoding-- if that were the case I wouldn't be able to play back smoothly.
Why have both? The firmware, being derived from what was a set-top box, has a lot of proprietary magic to make things work with displays, also specifically with things like the official RPi display.
I need to see if there are any RPI400 Chromium Video Acceleration Challenge players streaming on Twitch, maybe they'd have some clues. There are bound to be a bunch of useful weapons to beat this level but I don't know how to unlock them.
Then I upgraded to Raspbian buster to get a more recent version of Chromium since HBO Max was blocking me based on whatever version shipped with (buster minus one).
At that point I started shooting various startup flags at Chromium from the web of wizards' incantations until video started to play again. That including removing the 'f' per the incantation I mentioned. This cured video playback health but it put my mouse under some kind of latency spell.
I think I could beat this level with a drowsy mouse if only I could find increased magic for the subtitles somewhere.
Does anyone know if there is a Chromium arg I can use to just farm some magic?
Basically, disregard everything you’ve learned about KMS from the world of Raspberry Pi. It’s only relevant to the quirkiness of the Raspberry Pi’s half-closed ecosystem.
The F in fkms stands for “fake” because they created a quirky abstraction to fake KMS compatibility. The non-fake driver has been all sorts of problematic but it’s getting better.
But none of this Raspberry Pi specific weirdness should be extrapolated to the KMS/DRM subsystem in general.
Nothing about the Raspberry Pi at all is "normal," and it's a very useful filter for people who have read about the various deep weeds of firmware/OS development and those who have actually gotten their hands dirty on a Pi - because the first group will express admiration about the "open and well documented hardware," and the second group will laugh at the first group and start cursing the various non-and-poorly-documented nonsense that's in the Pis.
They're nice devices, and work well enough, but they are architecturally insane and unlike anything else except the previous Raspberry Pis that were hacked to to make the new ones.
The Pi1/2/3 are more or less an (undocumented) GPU with an ARM core bolted on the side. The GPU boots the system, allows the ARM core some access to the DRAM (not at the same addresses as the GPU sees things, so be sure which version of memory addresses you're looking at), and most of the hardware control is done by asking the GPU to please do this thing for you through the mailbox interfaces. In the same way that userland applications make syscalls into the kernel, the kernel running on a Pi makes syscall-like requests into the GPU to do just about anything.
The reason you see things like "All the interrupts are only ever on a single core" is because the interrupt controllers simply don't allow anything else. The Pi2 BCM2836 interrupt controller feels a little bit like an intern's project, using the BCM2835 bolted on the front as an input for the rest of the hardware. At least the Pi4 finally gets a GIC...
They're fine, sort of... as long as you're OK with the only documentation for some of the hardware being "This Linux driver works with it" and "Well, people have reverse engineered most of it." But they're absolutely insane, architecturally, they're poorly documented, and this doesn't seem widely known.
Is the Pi4 any different/better?
Raspberry Pis were initially just a way for the Pi foundation to help Broadcom sell SoCs nobody wants anymore but they still had lots of stock/tooling for. Sort of like "hybrid" bicycles are how bike manufacturers milk more money out of hardtail (non-suspension) MTB frames.
Very similar to previous-gen game consoles, e.g. the Wii U’s “Broadway”
https://github.com/librerpi/ https://gwolf.org/2022/04/how-is-the-free-firmware-for-the-r...
> Basically, disregard everything you’ve learned about KMS from the world of Raspberry Pi. It’s only relevant to the quirkiness of the Raspberry Pi’s half-closed ecosystem.
It's funny, I just posted a comment about being afraid to dig into KMS code for fear a wizard would show up and tell me I'm looking in a wrong or irrelevant place. Luckily, your comment preempts that process.
And also luckily, I've only thrown flags at Chromium provided by other wizards so far. So my ignorance about KMS remains pure. :)
> The non-fake driver has been all sorts of problematic but it’s getting better.
So... are there like two different teams, one Pi-specific 'f', one general 'not f', racing to see whose module will become stable first?
Do both teams stream on Twitch? If not they should.
OK, so first off, FKMS vs KMS.
KMS is Kernel Mode Setting. This is the kernel's framework for setting video modes, abstracting over HDMI/DSI initialization, timing parameters, etc. etc. across platforms.
FKMS is still KMS, but it's a special KMS driver for the Pi which sets up HDMI using the DispmanX API into the Broadcom proprietary OS that runs alongside Linux on the Raspberry Pi. This is nice because it's dead simple for Linux (Linux quite literally gets to say "yo, gimme a 1920x1080 layer!") but bad because it relies on a proprietary blob and the blob can't be extended.
Native KMS, on the other hand, sets up HDMI using native device access to the actual registers which affect the HDMI encoder, DSI link, and so on. This is more extensible and isn't closed source, but it's also mega complicated.
Now, here's the kicker on your Pi. Other commenters are correct that normally, KMS should just affect mode setting, not rendering. But, on the Pi, this isn't true. KMS and FKMS _also_ affect the way _layers_ are configured in the DRM (Direct Render Manager) subsystem.
Why? DispmanX isn't just a mode setter, it's also a scaling and compositing (layering) engine!
So, in the course of bypassing DispmanX, the DRM system in KMS mode now needs to act differently - it needs to configure 2D compositing planes through the KMS system instead of through DispmanX.
What's on a separate 2D compositing layer?
The legacy cursor.
Here's the switch where RealKMS is responsible for setting up the cursor compositing layers, a task which in FakeKMS was already handled by DispmanX:
https://github.com/raspberrypi/linux/blob/aeaa2460db088fb2c9...
https://github.com/raspberrypi/linux/blob/1fe617a917eac75800...
Now, _why_ this makes your cursor lag, I don't know, but that's the difference.
Clearly those commenters don't have much experience working close to the wire around GPUs then. Scanout used to be very closely tied to rendering, and in some ways it still is. You can't take an arbitrary texture in GPU memory and scan it out to the display.
Windows has this thing called VidPN to work around the dependencies between source and target modes which gets even more complicated with VRR and virtual resolutions (retina).
https://docs.microsoft.com/en-us/windows-hardware/drivers/di...
This has become a lot more important with smartphones than desktop systems. You now have often very different silicon for things like video decoding, graphics and scanout - and to achieve reasonable battery time you need common use cases like watching a video to not involve the GPU at all.
They're exposed as an independent engine on the GPU/SoC and can operate with most of the chip power gated.
The insight behind DrDc is that display compositing is higher priority than rasterization, so having both share work on the GPU main thread is bad for performance. DrDc is a large effort, but at the risk of oversimplifying, it's main purpose is to create a new GPU thread that the display compositor can use exclusively. DrDc is still rolling out on most platforms which may explain why it's disabled on your device, but regardless, it should not help with the video playback issues you're seeing.
You can pick up SBC-like computers with Intel integrated graphics and not have to deal with these problems from vendors' unsupported hardware, and there are some with AMD APUs. There are also SBCs that have PCIe slots that you can stick your own graphics cards into, but I'd stick with x86 machines because embedded ARM devices almost always require forked kernels that might not support everything you'd want them to, including your otherwise-supported graphics cards.
Some Mali graphics chips also work pretty well with Panfrost/Lima.
Personally, I won't be buying more ARM SoCs unless they start supporting SBSA and have mainline kernel support for all of their peripherals. As it stands, the only hardware that I'm aware of that has full mainline support for everything, including GPUs, are boards with Intel iGPUs or AMD APUs.
So, I really think they should take a few extra words and put this in either the introduction to the GPU Driver Developer's Guide or the first page in the section entitled "DRM Internals". They do that for KMS!
:~/
https://en.wikipedia.org/wiki/High-bandwidth_Digital_Content...
They seem to do go a good job updating major components for latest-1 release: kernel, Firefox, etc.
However, I noticed that say Chromium updates are not as fast or at latest version. So using Fedora that way might not be best choice.
To update to the latest Fedora versions I run a shell command and 30 minutes later I have the new version and I can hardly tell the difference.
I guess on some level we just prefer what we're used to.
But I seriously wonder about people who run Fedora desktops though. I installed Fedora 36 two days ago to try it again, and it's unusable. The scaling options are 100%, 200%, 300%, and 400%. I have a 43" 4k screen, so around 120dpi. 100% is too small, 200% is way too big. And gnome settings doesn't even have font size options. What the fuck.
gsettings set org.gnome.mutter experimental-features "['scale-monitor-framebuffer']"
The problem with it is that X11 doesn't handle the scaling well. Tend to get blurry fonts with X apps.Now that most applications are supporting Wayland this is less of an issue than it used to be.
It's not suprising if some level of indirection on top of it broke it in some environments though. Perhaps it has something to do with gnome 3? It works with gnome 2, kde, and all the minimalist window managers I've tried.
None of those try to "scale framebuffer" though, whatever that means. If it means what it sounds like, whoever implemented it is incompetent.
And I wonder about people who can't figure this part out.
> and now it's the unusable default gnome interface (so bad Red Hat made a shitty gnome 2 clone for enterprize customers).
Kinda odd how it's the most popular Linux desktop and you are the one that can't figure it out and also the one calling it shit and unusable.
Seems more like a personal problem than a Gnome problem.
TL;DR Most things are just "a little nicer" and a "little more sane" I found.
EDIT: I also run i3 so I'm not part of the gnome(n..m) "problems" etc. Can really recommend it !
For us, it wasn't worth the deeper investigation, so we went to just ensure simplefb (same spirit w.r.t. taking over the firmware frame buffer, but no DRM) is activated for now.
Albeit, on one machine we could also get simpledrm to work after a BIOS update, so I'm definitively interested in how that plays out with Fedora.
In any way, it's definitively a welcomed change, so if it plays out well for them, I'll try to switch again with our next Kernel jump.
> Development of Kmscon stopped in March 2015. There was a successor project called systemd-consoled, but this project was also later dropped in July 2015.
That doesn't sound very promising. Why can't kernel itself ship a more advanced terminal?
Probably because nobody wants to put opentype font rendering and input method support and various other things from a window system into the kernel? This stuff all has to be in userspace so you're better off just running a minimal X or Wayland server at that point.
uBlock Origin has kind-of-similar (but not quite) functionality where you can block by domains, but it is not as granular (it is called "Advanced" something IIRC).
Once upon a time, breaking userspace was a no-no [0].
You can still use DRM/KMS to drive "low-tech FB interfaces" and it's actually better because you don't have to deal with the weirdness of the fbdev ioctls.