Hacker News Reader

Comment by grishka | Hacker News Reader

Hm. But then wouldn't it make more sense to just stream the raw sensor data, which is 1 byte per pixel (or up to 12 bits if you want to get fancy), and then demosaic it on the host? Full HD at 30 fps would be 59.33 MB/s, barely but still fitting into that limit.

But then also I think some webcams use H264? I remember reading that somewhere.

View on HN

The pixel density doesn't generally refer to the density of the Bayer pattern, which can be even denser. Generally a cluster of four Bayer pixels makes up one pixel (RG/GB), but like most things in computing, the cognitive complexity is borderline fractal and this is a massive simplification.

> Full HD at 30 fps would be 59.33 MB/s, barely but still fitting into that limit.

It's not fitting into anything I fear, best case scenario the effective bulk transfer rate of USB2 is 53MB/s.

60 is the signaling rate, but that doesn't account for the framing or the packet overhead.

It would need a funny driver and since that stuff is big parallel image processing it's easy in HW but if someone has a netbook or cheap/old Celeron crap it would peg their CPU to do the demosaic and color correction.

> Full HD at 30 fps would be 59.33 MB/s, barely but still fitting into that limit.

That limit is too high even as a theoretical max.

You could do raw 720p.

I don't know where you get "1 byte per pixel" from. At minimum, raw 4:2:0 video would be two bytes per pixel, and RGB would be three bytes per pixel with 8-bit color depth.

When talking about digital cameras, each "pixel" is a single color sensor. Blame marketing.

Also 4:2:0 is 6 values per 4 pixels. 1.5 bytes per pixel at 8-bit.

You're talking about processed color frames. The GP was suggesting that the camera stream the raw sensor data, which doesn't have individual color channels, just a monochrome grid with 10 or 12 bits of usable data per pixel. A bayer filter[0] is placed in front of the sensor so that a given color of light falls on each cell. The USB host would be responsible for applying a demosaicing[0] algorithm to create the color channels from the raw sensor data.

If we take the AR0330 sensor used in the USB Camera C1[2] as an example, it has a native resolution of 2304H x 1296V and outputs 10 bits per native pixel after internal A-Law compression[3] for a total raw frame size of 3.56 MiB, assuming optimal packing. The corresponding image, demosaiced and downscaled to Full HD (1920x1080), in RGB with eight bits per channel would be 5.93 MiB.

[0] https://en.wikipedia.org/wiki/Bayer_filter

[1] https://en.wikipedia.org/wiki/Demosaicing

[2] https://www.kurokesu.com/shop/cameras/CAMUSB1

[3] https://www.onsemi.com/products/sensors/image-sensors/ar0330

> it has a native resolution of 2304H x 1296V

Seems to me like that kills the idea dead? GGP assumed 8bpp and that the raw resolution matched the output, and came out... well wrong (the effective bulk transfer rate of USB 2.0 is 53MB/s on a good day), but by just a few megs.

However the raw resolution is 40% higher than the final output, meaning even at 8bpp you're at 85MB/s and you've blown way past any hope of recovering via a few tricks. At 10 bpp you're above 100MB/s.

Native resolution at 10bpp requires 40% less data per frame than the final Full HD RGB output at 8bpp per channel (24bpp total), so it would represent some savings.

The problem is that neither format fits within the limits of USB 2.0 at 15 FPS or higher. To achieve a reasonable framerate you need to apply compression, and generally speaking you'll get better compression if you demosaic first.

Reply on news.ycombinator.com