Patents is often a concern. Maybe you think you're not violating any patents. But somebody with access to your code could find something that looks like a violation. Real or not, it can spell trouble for you.

Your code delivery could also contain parts that are licensed from other companies, limiting your options with regards to licensing.

One way to limit the problem is to put most of your "secret sauce" code onto one or more sub-processors embedded in the imaging hardware. The register bank of your hardware would only be accessible to these sub-processors, meaning that you don't have to publish your register definitions. Then you can deliver the sub-processor microcode as a binary blob (independent of the host OS) and let the kernel module itself be open-source. This approach is taken by at least some vendors.

I worked on a embedded hardware application using Linux and "cameras".

We (like many people) were using sensors from OmniVision[0] and there was a dedicated contractor who's entire area of focus (and life's work) was getting photons from the sensor of the camera into something usable by software. In our case that was an unholy combination of vendor software, custom code derived from very obscure NDA-only datasheets, and a bunch of custom plugins for gstreamer that could actually make use of the sensor, interfaces, and hardware encoding silicon.

It was one of the more eye-opening "WOW, so that's how the sausage is made" experiences of my life.

[0] - https://www.ovt.com/

The "3A" algorithms - Auto-focus, auto-exposure and auto-white balance are generally considered "secret sauce" by the various camera companies and thus closely guarded. The calibration process which is required to tune an ISP's pipeline is also very subjective and tied to the end use case. Having been involved with this area for the past couple of years I do believe that the openness of the raspberry pi platform (specifically with regards to the camera) is nudging the industry in the right direction.

It probably doesn't need to be in all cases, and for simple stuff like generic image processing (resizing/rotation), there's probably no "secret sauce" to worry about.

You could maybe technically argue things like denoising algorithms / foreground/background detection might be more secret as there might be competitive advantages there somewhere...