Hacker News Reader

Comment by londons_explore | Hacker News Reader

Smartphone cameras for the past ~8 years have been using multiple frames to improve noise performance, resolution and dynamic range. Sometimes as many as 128 frames are stacked, over many seconds, especially in low light. Gyro and optical flow data is also used to help align frames, and noise models to decide how to blend them (and avoid ghosting when things are changing in the frame).

How exactly does one fit 128 frames of lossless data from a 12 megapixel sensor into an 18 megabyte file?

Or is ProRAW not as RAW or as lossless as the name would imply?

View on HN

ProRAW is definitely not RAW, the amount of white balance correction you can do is far more limited. It think it's closer to capturing the different HDR layers to allow the dynamic range to be tweaked more in post.

Ex-Semi-Professional Photographer here...Yes, I think Apple is very misleading, if not being fraudulent, saying you can shoot in "RAW" format.

"RAW files contain uncompressed and unprocessed image data, allowing photographers to capture practically every detail they see in their viewfinder. The RAW file format stores the largest amount of detail out of any raster file type, which photographers can then edit, compress, and convert into other formats. Learn more about the benefits, drawbacks, and best ways to work with a RAW image."

If Apple is giving you JPEG-XL they are compressing the file, so it is not RAW.JPEG-XL is lossless compression.

Lossless compression shouldn't matter. That's why it's called lossless.

Lossless compression is still compressed. RAW photos are by definition not compressed. I do not care if you take out redundant bits, you are still taking them out.

"Lossless compression uses an algorithm to shrink the image without losing any IMPORTANT data." and the algorithm decides what is important.

Lossless formats can compressed: take FLAC as an example.

If it's lossless then there's zero loss and can be returned to 100% original file. However if algorithm decides what's important than it's by definition not lossless. Wouldn't be surprised to get something like that from Apple, they love to twist words into something they're not for marketing reasons. Like when they say that non-pro Iphone 15 has 2x telephoto while it does not have telephoto lens at all, it's just a digital zoom which all other phones have.

>I do not care if you take out redundant bits, you are still taking them out.

Why does it matter? Why does that make Apple deceptive? Lossless compression is as good as no compression, the only difference is the decompression time.

You could store the uncompressed RAW file on a filesystem with built in (lossless) compression, or you could store the losslessly compressed RAW file on a filesystem with no compression. Once you have opened the image in your editor or photo viewer, the bits in that application's memory that you are editing or viewing are identical.

Moreover RAW does not mean 'uncompressed'. It doesn't even mean losslessly compressed. Often RAWs use lossy compression. Both Canon and Nikon RAWs can.

https://photographylife.com/compressed-vs-uncompressed-vs-lo...

Have you ever zipped a file? Lossless does not mean data is lost. Lossless means data is not lost.

> RAW photos are by definition not compressed.

There is no such definition. Most cameras’ raw files nowadays are losslessly compressed. The compression being lossless means that the bits that were taken out can be reconstructed identically.

It seems you might be confused as to what lossless compression means?

Yeah, no, lossless compression is like zipping the file. The exact data can be reproduced by decoding.

No, lossless compression uses an algorithm to shrink a file without any loss. It can be restored to the same bitstream.

What you quoted is a confusing phrase: "visually lossless" which is in fact a lossy compression. It's like People's Republic which is neither a republic nor is it the People's, it's just a dictatorship.

Finally, JPEG-XL supports both lossy and lossless compression.

If you make a perfect physical copy of a picture, is it the same picture As the original?

You can not do that. It is literally impossible to make a perfect physical copy.

You can make a perfect copy of a file, however.

I am confused. I assume you use camera that does not compress raw files. which vendor camera is that? Nikon compresses raws, Canon compresses raws, Sigma compresses raws, Sony compresses and processes (see "star eater") raws, most cameras process known defective pixels and subtract dark frame from long exposures before saving raw...

Is it some kind of industrial camera you are talking about maybe?

on Nikon cameras you can choose between compressed and uncompressed raw files. So I’m assuming you haven’t been around that many cameras.

https://www.dpreview.com/forums/thread/4635816

You just said yourself that raw files cannot be compressed... now you say that Nikon has your permission to compress raw files? What is the point you are trying to make?

(Also, try reading the thread you linked and reconcile it somehow with your previous claim that losless compression of raw files is unacceptable)

> If Apple is giving you JPEG-XL they are compressing the file, so it is not RAW.JPEG-XL is lossless compression.

Are you unaware that most DNG files are compressed using Lossless JPEG? Including ones that are storing raw sensor data rather than debayered data like ProRaw?

Most DNG files, not all DNG files. In the past DNG files were always raw uncompressed files.

You can shoot real RAW with the iPhone hardware, just not with the system camera app. If you use something like Halide or Photon Camera, they have proper RAW options that give you a traditional noisy RAW.

“Noisy RAW”? What? It gives you a RAW file. Period. But this proves my point that Apple does not capture a RAW file, it is pretty highly processed. I want to do my editing, and I want to compress it my self if i want.

And I am talking about apple in this thread.

Apple does offer the ability to capture a lossless RAW file with raw sensor data, just not with their app. It’s exposed through their API for 3rd party apps to use. It’s harder to get a good result in low light or high contrast scenarios from a single exposure of such a small sensor, but many people prefer it to the processed ProRAW files.

The ProRAW format that the default app offers has a different use case: it combines the automatic advanced processing of multiple exposures into a file format that offers more flexibility for post processing than a standard photo does. With the new lossy compression option it gets more attractive for some users that are willing to trade in some quality for space. I don’t see the problem in offering that option next to the lossless compression.

Lossless compression is something I’m not going to argue about, as there is literally nothing lost by doing that and many professional cameras do the same thing, though with different algorithms. Well, unless your argument is that it’s not yet compatible with many photo editors, because that’s a fair point.

As an ex-professional photographer , you should be very aware that RAW does not mean uncompressed or lossless.

Many professional cameras offer lossy and lossless compressed RAW, and have for over a decade now and most are processed to some degree before it’s even written.

RAW hasn’t meant literally raw in a very long time.

Even the regular lossless RAW setting can sometimes be "not quite RAW" when the camera dynamically lowers the bitrate it captures at. My camera does anywhere between 12 bits and 14 bits RAW depending on the shooting settings. Many newer cameras have baked-in NR that cannot be disabled (e.g. Canon R5), even on the lossless RAW setting

ProRAW is just DNG with some additional metadata, and DNG can store actual raw sensor data, but Apple stores the semi-processed image after debayering and multi-exposure stacking yeah. The benefit is that it's 12-bit linear data so you get a decent amount to work with, and their automatic tonemapping is stored as metadata so it can be non-destructively tweaked or disabled in post.

Well, I suppose the less noise is in your image, the easier it will be to compress.

Maybe it’s just one frame per file? Also, you could probably achieve some pretty good compression with multiple frames because the frames are probably very correlated, right?

So if it's one Raw sensor frame, it will be noisy, and you won't get any of the computational photography benefits. It will end up looking like a photo from a 10 year old phone.

You will get a little compression storing multiple frames, but not much, because the photon shot noise is independent for each frame - and in lossless images, the noise (which does not compress according to information theory) often has more entropy than the ideal image (which compresses well).

Or you could store one post-processed frame, after you have combined 128 frames and done the computational photography stuff. But at that point, it isn't exactly raw data anymore.

"So if it's one Raw sensor frame, it will be noisy,"

That is not why noise is created in digital photographs. And Noise or Grain in a photograph is not always a negative.

"Noise in photography is the arbitrary alteration of brightness and color in an image. The onset of this random variation generates what is called “noise”or “grain”, which is basically formed by irregular pixels misrepresenting the luminance and tonality of the photograph. These pixels are visible to the eye due to their large size."

A signal RAW image does not have to have noise. Noise can be created by incorrect exposure and low light.

>Or you could store one post-processed frame, after you have combined 128 frames and done the computational photography stuff. But at that point, it isn't exactly raw data anymore.

Yes, and Apple should not be calling any images it captures from an iPhone "RAW".

> A signal RAW image does not have to have noise. Noise can be created by incorrect exposure and low light.

You always have noise. It's more noticeable in low light, but it never goes away. It's due to physics - there's real world noise in photons hitting the sensors and read noise in the A/D converters behind them. That's long before the image is captured into any format - RAW doesn't help you here.

There is always noise. In fact, in absolute terms, there is more of it as the amount of light increases. It just increases more slowly than the signal, so the signal-to-noise ratio increases.

Are there any cameras out there that output real raw data from the senors?

There's quite a bit of analog and digital post processing and I doubt you can bypass all of it

> Are there any cameras out there that output real raw data from the senors?

I suspect not, because the data output rate from a high end sensor can easily be 32 Gbps, and I don't think any phones have fast enough flash storage to write data at 32Gbps.

You have to process the data in mostly-realtime or lose the data. Storing it isn't an option.

> Storing it isn't an option.

Full sized cameras do use a buffer to deal with burst activity, take series if shots - process - save to slower storage. Also, modern cameras start switching to CFexpress/SDExpress (which are NVMe)

> Version 8.0 was announced on 19 May 2020, with support for two PCIe lanes with an additional row of contacts and PCIe 4.0 transfer rates, for a maximum bandwidth of 3,938 MB/s

Full-sized mirror/mirrorless cameras do that. Sensor is HUGE, optics is very fine and huge as well. Due to how universe works, you can't get similar results with tiny sensor and optics

> noise (which does not compress according to information theory)

That's true in theory/isolation, but in practice we know the noise has much lower maximum amplitude than the original image. I.e. if you take a difference between two frames, almost all of it will require less bits per pixel than the signal. (And that allows compression in practice)

Based on article [1], ProRAW is a half way between plain RAW output from sensor and baked in jpeg. It uses computed multiple-frame result, but stores more data, allowing you to recover details from shadows and do other things you typically use RAW for. Debayering is performed (nobody cares about that anyway), multiple frames are combined but higher bit-depth is preserved (12-bit instead of 8-bit), it also contains separate "tone map" which describes HDR gains (for each pixel?). Disclaimer, zero experience with ProRAW, just glanced at that article, but used RAW on DSLR in the past.

[1] https://medium.com/halide/understanding-proraw-4eed556d4c54

Reply on news.ycombinator.com