My worry is when this gets used for something forensic.

I imagine this will tend to reproduce things in the dataset, e.g. up-scaling blurry text may look like fonts that it has memorized more than the original. Or upscaling a feather will provide details like the feather of more common birds. Or upscaling blured out numbers will pick some numbers at random [1].

We need to make sure people don't rely on these details, e.g. in courts, HR reviews, when reddit sleuths try and investigate an incident, when someone looks for cheating partners etc.

[1] https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea...

Photography has always been more artistry than reality. From film selection, lens choice, framing and cropping, color correction, "dodging" and "burning" details in our out it's always been a compromise between what the camera sees (which itself may not be reality if you're using physical in-front-of-the-lens filters or other distortive techniques) and what the photographer want to convey. That's one of the reasons that Photographic Journalism often holds itself to ethical limitations with regards to how a photo can be manipulated after it's been taken. I'm almost certain that this kind of thing would at least walk that fine line if not fall right on over it.

Yeah, although technically the goal behind things like 'super resolution' is to add data that is not there anymore but probably was there before.

i.e. you had the original scene that was captured by a digital camera (a lossy operation) and then saved as an image file (often also a lossy operation), and then a tool like this makes an educated guess as to what information was lost in the 1st and 2nd steps.

Anyone who edits photos in Lightroom understands that modern photography captures both reality and artistry.

My goal when editing photos is to make them more clearly express how I felt or how I saw. This is often quite divorced from what shows up on the back screen of my camera.

A good experiment (which I cannot do): take a true photo of somebody, reduce it in size and then “super-resolve” it. How much must you reduce in order to produce a different “person” (or a non-person).

It's always been more like painting than anything resembling a truthful representation of reality. In fact, the first ever photograph [1] much more closely resembles a painting than anything else, which is ironic.

All advancements have simply given us more control in how painterly we render our photographs, but have never _really_ brought us closer to the truth.

[1]: https://en.wikipedia.org/wiki/History_of_photography#/media/...

It is sort of. Their argument is that by taking into account the subpixel pattern of the sensor they can actually extract more detail than is readily visible in the picture.

Basically you can imagine that the blue subpixel is always to the top-left of the pixel. If you shifted the blue down and right one half a pixel you would have a more "accurate" production. In this way you can add a new pixel with the blue value closer to the right spot, then interpolate the blue of the original.

Of course you can also do logic such as detecting lines of different lightness and applying those on top.

So yes, especially with their machine learning they are adding new detail, but that is also likely some detail that was already there, but could not be conveyed with with the lower resolution. I wonder how different this would be from the simple approach of realigning the subpixels on a higher-resolution image and interpolating the "missing" subpixels. This approach may look better but wouldn't add any data.

Ansel Adams answering a similar question https://youtu.be/Ml__B0l9GIs?t=1514

Presumably you go from picture to painting the moment you open photoshop.

Photography for record keeping/science and photography for aesthetics/artistry diverge long before you get to techniques like super resolution. Which is still a fuzzy boundary because anyone who has taken a picture including a sunny sky can tell you raw photos generally don't capture how it looks to your eyeballs.

Not true, there’s always far more data in an image than a quick glance can decipher, astronomy field obviously pioneered numerous methodologies over the decades (many of which biologists do more shittyly, speaking as a biologist who did so). In the end I’d argue a lot of these “super resolution” methods are just glorified dexonvolution, not that it’s a knock on these methods but apparently it’s not cool to call deconvolution deconvolution anymore.