Many SLRs are already well supported, though the open source stuff doesn't focus on low latency conversions, which is needed for viewfinders, or focus control, etc.
I did a side-by-side comparison of a Micro Four Thirds camera (4/3" sensor) and an iPhone SE (1/3" sensor) and the performance was... pretty much the same.
And I'm not talking about some ML interpolation wizardry or automatic face beautification; I was photographing barcodes and testing how many were readable in the resulting images - hardly something Apple would have specially optimised for.
The iPhone has a much smaller sensor, a much smaller lens, costs less, and manages to pack in a bunch of non-camera features. To be competitive in the modern cell phone market your camera has to be straight up magic.
But when I say the performance is good, I'm not just declaring the images good because the portraits have simulated bokeh, or face-detecting autoexposure, or image stabilisation, or tasteful HDR, or a beauty mode that airbrushes out blemishes and makes photos of sunsets really pop.
Even in applications where none of those features come into play, the iPhone, can still go toe-to-toe with cameras with much larger sensors.
Did you compare raw sensor output, or post-processed?
Big sensors capture more light and have more bokeh. With enough light, the first doesn't matter, and bokeh is not a thing for QR codes.
If you didn't have enough light, then it's probably the question of how denoising was done, and what details have been guessed by fancy algorithms. Geometrical shapes are easy to guess, but when I look at pictures of landscapes, they typically devolve into a painting after 1000×1000px if taken on a phone.
If I have to rewrite stuff for low latency, I'd rather start it as an independent library so that other projects can reuse the code.