Raspberry Pi 4: Hardware accelerated video decoding (GPU) in Chromium
lemariva.com
lemariva.com
[1] https://www.hackernoon.com/can-a-pi-4-replace-your-home-thea...
I am slightly miffed at that. I'm running some docker containers and a Sambashare NAS on a Raspberry Pi, are you claiming what I have is not a server?
If you have ecc or n+1 psu support at home, I ask - do you have cardkey access to your front door? Do you have a fallout shelter and a food stockpile for one year in the basement?
I also helped the Raspberry Pi Foundation around 4 years ago building and packaging their patched version of Chromium. What they essentially did was directly modify the Chromium source to add HW decoding specifically for the Pi with MMAL. Then, an extension would be auto installed to enable H264 on YouTube instead of VP9. This was released back when they launched the Pixel desktop.
This obviously had some problems, because the patch needed to be manually changed for every release of Chromium, which come pretty often. It couldn't also be easily upstreamed. Looking at it now, it seems it hasn't been updated in about a year, and I'm curious what the Foundation's roadmap for this is in the future, after having been out of the loop for so long.
There is the BSP, id est Rockchip's hacked together fork of the 4.4 kernel, which works. But on the mainline kernel h264 is just now getting added (maybe in 5.10), and userspace is still going to take some time to adapt.
I really hope that one day I can actually use the Pinebook Pro webcam for video chats without my CPU spinning while trying to encode (at least) one stream and decode N streams.
Only really true for VOD, and only really true for channels with 300k+ subs.
I haven't tried it on a pi, you might have to specify --hwdec or some variation of it to have really smooth playback, otherwise it's as simple as `mpv "url"`.
81 is three weeks old & is the first release that will have Linux video gpu support enabled by default. source: https://9to5linux.com/firefox-81-enters-beta-gpu-acceleratio...
note that this is firefox at large. no idea whether the pi supports the standard va-api acceleration interface or what other potential blockers the pi has.
Question-- how exactly does one check that gpu-acceleration is on in FF? I'm running nightly on a NUC with Intel Iris GPU and it seems to be faster than LTS from Debian Bullseye, but I'd like to be sure...
More info here:
https://wiki.archlinux.org/index.php/firefox#Hardware_video_...
Maybe firefox whitelisted only h264 for hw decode?
[Child 78229: Main Thread]: D/PlatformDecoderModule VA-API FFmpeg is disabled by platform [Child 78229: Main Thread]: D/PlatformDecoderModule Sandbox decoder rejects requested type
Same in Nightly 82.0b9.
This is on a Dell XPS 13 amd64 with Mesa Intel® UHD Graphics (ICL GT1) running Ubuntu running Wayland.
I haven't tested my RPI4 yet. But if Intel graphics aren't a "supported platform," it's difficult to imagine any graphics under Linux that would be.
Didn't test trying to force-enable because we're talking about supported platforms.
Whether that translates to actual video decoding acceleration on any platform depends on availability of libva backends. And performance (or power consumption) may still suck, if you can't use the frame buffers directly for output to some HW overlay.
Linux kernel APIs are fairly straightforward, and easy to use, if you use them directly from some simple app. But making it work across all the abstractions from browser, to compositor, to display server, via ffmpeg, via libva, etc. is not easy at all.
Another possibility is just that nobody has gotten to it yet.
I assumed Broadcom would be happy to have a platform as popular as the Pi using it's SoC. There must be a lot of companies out there prototyping on the Pi (because it's probably one of the easiest dev-board to get started on).
The Pi is a rounding error money wise.
I mean, I get that they supply parts for specialized devices, but why would a company pass up on the opportunity to reap as much goodwill and brand recognition as the position of "the people that made the Raspberry Pi economical" could engender? I mean think about it for a minute. This is this generations 486 or Commodore or VIC-20.
I scratch my head at trying to imagine the thought process of those in management to completely throw away this opportunity unless they really have something big to hide.
Many millions of Raspberry Pis are sold in a year. They aren't Broadcom's highest priority but the foundation is a legitimate customer now.
It's way too high level. Broadcom would be glad to sponsor something like Verilog/VLSI engineering tools. Maybe like an FPGA board that implements certain Broadcom proprietary instruction sets and lets people use that as an onboarding ramp to get them inducted in a "first hit's free" kinda way.
Broadcom customers buy chips in order to build other electronic products. Not to learn or program python or even C/C++. The original intent of the Rpi as an embedded controller is closer to what Broadcom wanted to do, and likely was how RPi talked them into agreeing with the whole thing in the first place.
That market position has been taken over by the arduino family though, so the positioning of Rpi within the Broadcom general strategy isn't well matched.
The Pi itself was a herculean feat just to get broadcom stop suing consumers for using the chips as an individual consumer.
Broadcom itself is about as anti-user as they come.
What they really want to do is to sell you the features - you want decoding that's already on the chip? Sign NDA and pay them a few million dollars for each seat / license and they'll let you use it.
- The more comprehensive your documentation, the easier it is for your competition to get a complete read on your product and it's capabilities. Which makes it both easier for competitors to strategically outcompete you, as well as counterfeiters to more accurately counterfeit your chips.
- The more precise your documentation, the more stringent your customer's expectations become. Poor documentation can act as a form of liability insurance - if someone uses a feature and things get weird, you can claim their use is out of spec. Harder to weasel out if things are clearly documented.
- The more expansive your documentation, the less dependent your (potential) customers will be on actually talking to you when evaluating if your product is fit for their use. I could see this being a negative in this industry - Feature X may be fine to use, but introduce intermittent instability if used for extended periods of time simultaneously with Feature Y, which can be compensated for via Z but doing so shortens expected lifespan by 30%. Wargaming out every possible contingency ahead of time would both be extraordinarily expensive, and result in documentation that would overwhelm the people using it. But not doing so while providing otherwise expansive/comprehensive documentation may lead to a false sense of understanding from your customers such that they write you off when your product would have in fact been a good fit.
- A corollary to the above - poor documentation trains your (potential) customers to just reach out directly and have a conversation about it. Which provides the chance for you to truly evaluate their situation and provide a more considered response that takes into account their use/circumstances. Which can either be using the product they may be familiar with, or suggesting a different one that's more appropriate that they may not have thought of.
- Ditto the above for implementation. Directly supporting them during their implementation can be more effective than them relying on generic documentation, since you can provide more contextually relevant details/instructions/direction. Which cycles back to the second bullet above, where you have the opportunity to provide information that steers customers in the right direction, addressing potential issues preventatively rather than playing damage control if their use wasn't completely aligned with the documentation they blindly followed. You can also directly put the costs of this work against the customer, rather than eating it via the comprehensive wargaming scenarios mentioned above. Which sounds harsh, but is far more efficient for everyone involved. Customers with a cookie cutter use case get cheaper per unit prices since you didn't inflate the amount of QC and experimentation required to flesh out the documentation, and you didn't waste time and resources chasing down contingencies that were never actually applicable to any of your customers.
So it sucks for a subset of people - hobbyists who aren't direct customers in the first place, cash strapped hardware shops who can't pay for that support, and the engineers at BigCos who aren't willing to pay for that support. But can still be rationalized when you consider it's an absolute non-issue for their primary target market, while also indirectly benefiting everyone else via the lower unit costs afforded by this mentality/approach.
But the more intelligence a Thing has, the inversely worse the situation becomes: good luck finding, for example, datasheets for a USB controller for Intel HDA codecs (if anyone has a hint on these, email is in profile)... and it's outright impossible to find anything for more complex stuff like Thunderbolt controllers, much less CPU / GPU / SoC chips. The closest a hobbyist comes across to real performance beyond RPi is COMexpress, which is not an as open standard as many would like plus the connectors cost a fortune...
And to top it off: even for components that have datasheets... these lack licensing information, which means it's technically illegal to put a pcb design based on them as open source.
Seriously I'd appreciate it if our governments could do something about that situation. The creativity decent rules about datasheets could unleash...
Except everything you mentioned above prevents new competition from ever getting a foothold to drive down prices. Everything you just described is. only good if your end goal is to be an immortal, indispensable fixture of the economy, and I'd even go so far as to say the practices you endorse as strategically sound are (or should be) explicitly considered examples of strategies which are anti-competitive, anti-consumer, and should entail strict liability with a punitive consequence for the company involved if they utilize those practices for the reasons you state as a cost for and any tragedy results from the opaque nature of their dealings with the public. Nothing short of undesirable results and wasted effort is inflicted on the rest of society; which at the end of the day is mired in the quest to efficiently allocate resources, and further the State of the Art and the frontiers of human knowledge.
You'll get the same exact things you counted as benefits for doing things badly simply by being proactive in communication with clients. and reaping the fruits of consumer and investor goodwill. You'll discover new edge cases and directions for truly novel developments when your customer tries to use your thing and, darn it, I need something just like it but with X. Hardware can be far better characterized, and the ability of the public to identify "the right tool for the right job" greatly increased. With more goodwill comes greater leniency, more willingness to allocate capital for novel problems, and a much more accommodating environment when it comes to risk.
It's the difference between being a long living parasite, and becoming a household name. A vampire may live forever, but eventually somebody is getting a Stake and some garlic. Start acting like you've got something to hide, and people will go out of their way to do away with you. Free everybody from being helpless to help themselves though? That gets songs, monuments, and a chapter in the history books about how awesome and pivotal you were.
Goddamnit, don't you want to live forever?
https://bugs.chromium.org/p/chromium/issues/detail?id=463440
from the article, this seems to require setting up an advanced video card config on the pi side & allocating 128mb ram. that used to be a significant portion of ram. rpi3 came with 1gb.
RetroPi + anything from 2D Nintendo consoles, I'm personally a fan of the GBA metroid games. Some other good ones are Fire Emblem, A Link to the Past, Final Fantasy, Mario (1/2/3/World). If you email me, I can help with sourcing ROM's.
As for more modern games, Hollow Knight is a masterpiece. A SNES controller has _almost_ enough buttons, but I had to remap one control to the spacebar.
There was an extension added in an attempt to fix this situation, IIRC it's the xdbe extension, but I don't believe it gained much traction.
If you look at the implementation of xcompmgr and the extensions introduced to make compositing possible, namely Composite and DAMAGE, they too don't have any frame-oriented coordination.
So it's just more racy uncoordinated capacitance in the immediate mode rendering pipeline of X. Your X requests logically just throw things on the screen as they occur. There may or may not be a compositor in the middle of that. When the compositor is scheduled and how much of your X requests sitting in the queue become processed as part of the same timeslice is fairly non-deterministic. There's unpredictable scheduling occurring at the kernel level, the compositor process's event handling, and the X display server's client multiplexing level.
There are some good reasons for a replacement like wayland, this area is definitely one of them.