IPU6 camera support in Fedora 41

Note: I'm wrong, surface support for ipu6 isn't there yet, see https://github.com/linux-surface/linux-surface/issues/1516 (via https://www.reddit.com/r/SurfaceLinux/comments/1fwyzu1/comme... )

My understanding is that it's not the camera that's integrated, but the ISP - image signal processor.

Cheaper camera hardware, more advanced software with computational photography?

Doing the image processing in software would use too much energy. The IPU6 is a PCI device that does complete image processing, handing fully baked frames to the system. If you wanted to do what the IPU does but on the CPU, you'd need to use the vector units and they use a lot of power.

USB cameras are interfaced over USB 2.0 because USB 3.0 is extremely difficult to make work from an EMI perspective running through a laptop’s hinges and lid. USB 2.0 means heavy compression to get 1080p60, and on pretty mediocre ISPs in those low cost camera controllers.

MIPI is easier from an EMI perspective and lets you use a much better ISP on the processor.

Making USB webcams that fit into the bezel of a laptop screen means they are basically universally terrible. In part this is because there really aren't many "native" USB imagers, almost all of the sensors are natively MIPI CSI devices, with a MIPI-USB interface that performs limited image processing and compression to present the UVC interface. Given the size/power/thermal constraints of a laptop bezel, there's very little you can do. If you compare laptop webcams to phones with similar size constraints on the sensor/optics but much more capable image processing units, phones do a much better job. Even if you have much more processing power available in a laptop CPU/GPU than a phone, once the camera feed has already been processed and heavily compressed there's nothing you can do to get that information back.

Historically, PC processors didn't have a native MIPI interface, and laptop vendors wouldn't have accepted the costs of adding an additional image processor to the motherboard or the effort to standardize a new camera interface. The first PC processors to support MIPI may have been Intel's "Silvermont" mobile/embedded processors in 2013, but didn't come to Intel laptop SoCs until some "Skylake" models with IPU3 then became fully mainstream with IPU6. Now, direct-to-SoC MIPI plus on-SoC image processing unit is widely available. Additionally, in the intervening years MS broke any all-cameras-are-UVC assumptions by using MIPI cameras on Surface devices and thus getting the software interfaces ready.

Nobody is saying other distros won't. This is just talking about it in Fedora and what it took to make it happen.

For high-level summaries, I actually appreciate such framing because the Linux ecosystem sorely lacks the coherence found in other platforms:

"USB4 V2 is supported in Windows 11 24H2"

"The Translation API is now available in macOS 15"

"To use the built-in camera in these specific laptops, you need Linux kernel 6.10+, libcamera 0.3.1+, pipewire with some downstream patches, Firefox 116..."

You get the idea. For users and application developers, "Fedora 41" represents a _coherent whole_ (roughly) and is a more productive subject to center discussions around.

For me it was for a bit, but it was very fragile, then eventually broke and didn't recover.