Hacker News Reader

Comment by dlivingston | Hacker News Reader

The author notes that 'But, there is no such standard...[a]ll real-world RAW processing programs have their own ideas of a basic default state to apply to a fresh RAW file on load'.

While I can accept that there isn't a universal RAW -> RGB standard, it seems strange to me that 'compute how this photo should appear' is left as an exercise as a reader.

Photographers often view their work as a form of art, and artists are very particular about even the smallest details of their work.

Why, then, would Nikon, Canon, and especially Leica, not have their own definable standards of how to process RAW photos for their particular cameras?

View on HN

A RAW file is a description of the light captured by the sensor. What good would such a standard do? The RAW file is not supposed to be a finished work of art, it’s designed to be processed.

Interestingly, some major photo competitions require 'original' RAW files to be submitted along with processed jpegs. The idea is that they can check that the image hasn't been modified in a way that breaks the rules.

Of course, software people know that it would be tedious but technically straightforward to concoct a fake RAW file from another 'raw' format such as a PPM image. The stakes in these competitions can be quite high so it's a bit disconcerting that they rely on a presumption that converting RAW files to editable images is a one-way process.

No - it's not straightforward. How the light strikes, sensor imperfections, patterns unique to each camera body even, thermal noise. It's very very hard actually. It's one thing to doctor a finished image - it's another, in a forensic level really, to doctor a RAW file. You can make one which on its own is syntactically correct but which does not show all the telltale signs of being shot in actual hardware, no patterns or imperfections etc. This is trivial.

Next level is to make it look plausible on its own, with plausible imperfections. This is a bit more involved and tedious.

The really hard part is to make it look like it came from the exact same individual purported camera body the photographer allegedly used. This is really hard.

An analogy would perhaps be taking a picture of a room:

Now, build another room that would make the same picture. Make sure everything in the room is plausible yet renders exactly the same picture as the first room. (The room is the RAW file.)

Edit: (I'm not saying all the competitions have the time and know how to verify a RAW file.)

Couldn't you just take whatever image you want to convert to a RAW file, make a really large high quality print out of it, and then take a picture of that? I would imagine that with the right lighting setup (and a really high quality printout) it would be pretty hard to tell that it is a picture of a picture.

Nope. Light is a 5D wavefront. Each x,y,z of origin in the real world is actually a sum of photons from many directions, scattering to many other directions. So you have two angular components. At times, phase matters too, and frequency, so each photon needs perhaps 7 parameters to capture its state, (x,y,z,θ,φ,ψ,λ).

On top of that, you have diffraction effects in the optics, chromtic aberration, and even some birefringence effects.

Algorithms exist which can detect if an image has been cropped, and what region of the original it came from. I'll leave as an exercise to the reader how that works :) hint: keyword is "media forensics"

The scenario I'm talking about is not synthesising a RAW file from scratch, but being able to freely edit it and save the result back into a RAW file that looks "original". I don't think you've made any statements that indicate that doing that would be complex.

Again, this is not about faking a RAW file from scratch, just being able to edit it in place.

One potential complication just occurred to me. If, (IFF!) each sensor site has a somewhat unique non-linearity in any way, you can't just go around in the image and change "pixels". You have to adjust them in a way that would not change the patterns normal to that camera for that sensor site and for neighboring sensor sites which could have been affected by the same light.

Oh, that I have too little insight in. I could speculate that going either way. Would be very interesting to know for sure!

Competitions, when an image is contested, sometimes ask for images made before and after the winning image. If it isn't a studio image, the time-series is hard to fake.

A similar misperception is common for PDFs.

The RAW file is analogous to the latent image on film. It's before most all developing and processing. A big difference is a metric f ton of non-linearity in film, and for practical purposes the RAW file contains linear data.

Adobe Digital Negative (DNG) [1] is that sort of standard (sort of) and there are manufacturers who use DNG as the RAW format in their cameras. Leica tends to use DNG as it's RAW file format. [2]

[1]: https://en.wikipedia.org/wiki/Digital_Negative

[2]: https://www.bhphotovideo.com/c/product/1082939-REG/leica_108...

> Why, then, would Nikon, Canon, and especially Leica, not have their own definable standards of how to process RAW photos for their particular cameras?

Pretty sure they do. Aren't those the manufacturer-specific import profiles in programs like Lightroom?

Nikon and Canon always try to push you to use their own software.

Phase One, Adobe, Photo Mechanic etc all reverse engineer the manufacturers formats, which is why the results differ.

Anecdotally, Capture One is (for me) way more responsive when working with Nikon .nef files than with Fuji .raf.

But are those written by the camera manufacturers?

I think your premise is a bit faulty. A photographer basically never gives anybody a raw image unless they’re giving up control of how the final image looks to somebody else (as some other said, there are some special circumstances, such as competitions - but this is not for anybody to use for presentation, just ensuring that the image hasn’t been altered in ways not allowed - i.e compositing multiple photos to fake some amazing occurrence).

Processing the raw file is almost as much a creative art as composition and photography itself! So if a application processes raw images in a way the photographer likes better than the algorithms another app uses, then all the better. They’re going to export it in the end anyway.

The camera manufacturers of course do release specifications and probably proprietary code for processing raw images from their cameras to companies like Adobe to use in Lightroom under NDA, which I assume they use as a basis, but all these apps have their own ways of doing adjustment and extra processing too.

They do to an extent. It's just that for some camera manufacturers, it's only accessible in-camera (i.e. shooting in JPG mode), for others, they do have a separate RAW converter you can use on your computer (Fujifilm does, unsure for other manufacturers).

Canon can't make Adobe or Phase One use their algorithm for decoding RAWs though.

They could not license the decoder for them to use in their respective software.

> and artists are very particular about even the smallest details of their work.

That's not necessarily true and paints way too broad a brush of artists and art. Art and artists need not necessarily be concerned about the smallest details. Particularly if it is an intermediate medium which never is shown to the art viewer/participant.

I believe nikon actually encrypts their raw file format. (or parts of the format)

Reply on news.ycombinator.com