Samsung “space zoom” moon shots are fake, and here is the proof
old.reddit.com
old.reddit.com
Sensor quality in phones goes down, AI makes up for it because good sensors are expensive, but compute time in the cloud on Samsung owned servers is cheap. You take a picture on a crappy camera, and Samsung uses AI to "fix" everything. It knows what stop signs, roadways, busses, cars, stop lights, and more should look like, and so it just uses AI to replace all the textures.
Samsung sells what's on the image to advertisers and more with the hallucinated data. People can't tell the difference and don't know. They "just want a good looking picture". People further use AI to alter images for virtual likes on Tiktok and Insta.
This faked data, submitted by users as "real pics in real places" is further used to train AI models that all seem to think objects further away have greater detail, clarity, and cleanliness than they should.
You look at a picture of a park you took, years before, and could have sworn the flowers were more pink, and not as red. You are assured, by your friend who knows it all, that people's memories are fallible; hallucinating details, colors, objects, sizes, and more. The image, your friend assures you further? "Advanced tech captured its pure form perfectly".
And thus, everyone will demand more clarity, precision, details, and color where their eyes don't remember seeing.
Another thing, pictures for proof and documentation, maybe not when they're taken but after the fact, for historical reasons, or forensics.. We can't have every picture automatically compromised as soon as it's taken. (Yes, I know that photoshop is a thing, but that's a very deliberate action, which I believe it should be)
That's me. I'm a lousy photographer, as evidenced by all of the photos I shot back when film actually recorded what you pointed it at. My photography has been vastly improved by AI. It hasn't yet reached the point of "No, you idiot, don't take a picture of that. Go left. Left! Ya know what, I'm just gonna make something up," but it should.
I imagine there will remain a use case for people who can actually compose good shots. For the remaining 99% of us, we'll use "Send the camera on vacation and stay home; it's cheaper and produces better pictures" mode.
This is fine, and I can take good shots but at the same time? I only care about this level of shot most of the time too!
But then instead of a 20MP image, which:
* takes more space, and ergo, more flash drive space
* more space to store, to backup, to send
* is made 20MP by inserting fake data
Why not have a 2MP image, which is real, and let people's end-use device "fix" it? Because all that post processing can be done when 2x or 4x the view size, too!
Because advertising.
And that's sad. We'd rather think we have a better pic, and destroy the original.
And the space thing is real. Because, that same pic gets stored in gmail with 20 people, backed up, kept in all the devices, and so on!
And the LOL of it all, is that I bet when it is uploaded to facebook... it gets downsized!
edit: in fact, my email app allows me to resize on email, so I downsize that too! Oh, those poor electrons.
There's nothing wrong with that until people start assuming they're lossless (like Huffman encoding) which they definitely are not. Unfortunately the general public doesn't understand the difference.
Yeah I've a phone with a great camera, nature shots are great, but people don't like themselves in these photos. When pressed they talk about the defects on skin and theets and eye position... Their phone beauty filters created in their mind a fake mental image of themselves and they dissociate from their real images.
It's weird. My mom brand fidelity is because Huawei specific algorithm is part of her self.
So, we’ll see what people say in a year.
[1]: https://www.wired.com/story/ai-has-a-hallucination-problem-t...
Isn't it a good thing for privacy?
Just take the story above with one more minor step: You snap a pic of the park, briefly glanced at it to make sure it wasn't blurry (which the AI would have fixed anyway) or had an ugly glare (it did, the AI fixed it) or worse a finger (the AI also fixed that).
You're satisfied the image was captured faithfully and you did a good job holding your plastic rectangle to capture unseen sights. You didn't look closely enough to notice all the faked details, because they were so good.
This fake moon super enhance? It already proves people will fall for it. I could easily see people not realizing AI turned the flowers in the picture more red, or the grass just a little too green, etc.
The story has to get stretched a lot to imagine a total dystopia, but its possible.
People like digital mirrors for all their benefits: lighter, more features, music and weather all on one "mirror". You can have voice chats, see how you might look with various makeup/styles/haircuts. This digital mirror gets so popular, and (undermining myself here a bit) high enough quality, that people want these new cool wall screens instead.
And you betcha, AI is of course going to be added. Take insta pics without needing to hold a phone, then apply filters all in one with your voice! Some people take dozens of photos, forgetting briefly how they look.
Now, people are used to every day seeing themselves in the digital mirror with minor touchups for how they would look if they used some sponsored makeup. They used to do it daily so it would match, but kept forgetting as it always showed the improved version (with a small icon saying, you are 40% matched to the predicted image!).
Some 20-something walking around in Chicago passes The Bean, and realizes they don't look quite the same as they did in their home mirror. They take out their phone to take a pic of themselves, which is of course synced to their mirror with the same "makeup enhancement suggestions", still warning them it doesn't match.
They put away their phone, confident in the knowledge The Bean is just a dirty and distorted mirror, which is why they don't look as good. The camera has always been trustworthy. Why doubt it today?
(again, fun story, I don't think this is likely. Just plausible for some people).
The contrast between what my camera raw with a stock profile puts out and what my iphone puts out is striking, and it's very clear the iphone's version of reality is optimized for Instagram and maximum color punch at the cost of looking real.
Thing is, that's what people like. So that's what we're getting.
Reality be damned.
Silicon Valley is pretty famously nice out most days. (Except for all those old strip malls in the way of the nature.)
https://www.destinationtips.com/wp-content/uploads/2018/10/P...
https://smarthistory.org/wp-content/uploads/2021/10/Giza_pyr...
https://thumbs.dreamstime.com/z/view-side-slums-egyptian-pyr...
My point is that there never has been, cameras have always let you adjust the shot - including "how red the flowers are" by changing light source, film type, shadows, which contrasting other colours are nearby, etc.
With old-school photo processing--yes, in a darkroom--you could achieve unrealistic results. But it was a choice. That's not what you got when you sent your negatives to the Costco to get printed. That's akin to the results I get when I use my camera, especially when looking at jpgs straight-out-of-the-camera.
In contrast, we get modern cellphones doing incredible processing to almost arbitrarily replace content with what some algorithm feels you'll like better, whether or not it resembles reality.
I lament that it usually resembles beginner photographer work, where they've just discovered HDR tone-mapping, local contrast, global contrast, saturation, sharpening, and smoothing filters, and promptly slam every single one of them to the stops. I did it, and now I recognize it when I see it in cellphone pics my friends send me via imessage.
Been there, done that. I recognize the stigmata of saturation slammed to the stops and excessive use of HDR and local contrast.
The default is excessive editing now, probably because it helps cover up limitations of tiny sensors, small optics, and poor exposure due to poor technique.
It does have higher than real contrast, but that's because images are 8-bit - if you don't try to fill the range, it's going to look low quality with banding artifacts.
https://9to5mac.com/2023/01/06/mkbhd-post-processing-ruining...
Larger cameras don't do that specific one as much ("dynamic range optimizer" or HDR programs do some of it), but they do care about skin in white balance and autofocus, and then photographers care when they're setting up flashes.
It's somewhat overtuned here though.
I generally just burst-mode-scan an area or scenery location and later that night, or when I add to Strava or wherever, I have an old school contact sheet (but with 60-80 images per thing) to look though. Then narrow it down to 5-10, pick the one or two I like best and discard the rest.
as AI tries to infer images of people where they aren’t really present.
The details don't quite correspond to reality; to get the framing right the AI inserted a tree branch where there wasn't one, or moved that pillar to the left to get the composition lined up. But who care? Gorgeous photo, right?
And the thing is, I don't think anyone would care. You'd get the odd weird comparison where two people take a photo of the same place and it looks different for each of them. And you'd lose the ability to use the collected photos of humanity to map the world properly.
I think it's fascinating. Reality is what we remember it to be. We can have a better reality easily ;)
It's the future. Something hit your self-driving hover car and left a small dent. To get your insurance to pay for fixing the dent you have to send them a photo.
Your camera AI sees the dent as messing up the composition and removes it.
Your insurance company is Google Insurance (it's the future...Google ran out of social media and content delivery ideas to try for a while and suddenly abandon so they had to branch out to find new areas to try and then abandon). Google's insurance AI won't approve the claim because the photo shows no damage, and it is Google so you can't reach a human to help.
Cue Paris Syndrome, because expectations will also be of a better reality. Then you go somewhere, and eat something, and experience the mess that actually exists everywhere before some AI removed it from the record.
My worry about these features becoming commonplace is that if everyone just leave those features enabled, we would end up with many boring photos because they all look similar to each other. The current set of photo filters, even though they seem to be converging on particular looks, at least don't seem to invent as much detail as pasting a moon that's not there.
Edit: And the lenses on these are not your granddads computar 3-8/1.0, either. Most of the CCTV footage we see just comes from old, sometimes even analog, and lowest-bidder installations.
Edit: fixed the author's name. Cannot find the exact story though.
What actually happened was that a police officer testified that using pinch to zoom on his iPhone he saw kyle point his rifle at the first person assaulting him. Mind you we were talking about a cluster of around 5px. The state wanted to use an iPad to show the jury using pinch to zoom that same "evidence" because using proper zooming without an unknown interpolator algorithm the defense using an expert witness showed that this was not the case. No one in that courtroom understood the difference between linear and bicubic interpolation.
The defense did not understand it either so they tried to explain to the judge that the iPad Might use AI to interpolate pixels that aren't there and that the jury should only use the properly scaled video the court provided not an ad hoc pinch to zoom version in an iPad with unknown interpolation.
Thankfully the judge told the state to fuck off with their iPad but the mainstream media used the bad explanation of the defense against kyle when the reality was that the state basically tried to fake evidence live on stream using an iPad to zoom in.
BTW I'm German so I don't have a horse in the political race but I watched the Trial on live stream and saw the fake news come out while watching
This is what the defense actually said: https://www.youtube.com/watch?v=sf7xCMFBv5c
And here's the expert witness the defense brought on: https://www.youtube.com/watch?v=1GhsbizmfMs
I watched the testimony, it was accurately characterized by the person you replied to, and it was a good argument for the defense to make.
That being said, one could interpret the top comment that way.
It is all faked by AI well.....
Why do you think the stock camera apps usually get better results?
How could a phone's smaller-than-a-thumbnail lens setup ever get as much light as a proper camera's?
So a future where reality and history are subtly tweaked to the specifications of those willing to pay…
I'm working close to HW and I actively use the camera/picture and videos for future reference and debugging. It's small, fits in your pocket, and the bloody thing can record at 240fps to booth!
Until you realize there's so much post-processing done on the images, video and audio you can't really trust and can't really know if you can turn it all off. The reality is that if you could, you'd realize there's no free lunch. It's a small sensor, and while we had huge improvements in sensor and small lenses, it's still a small sensor.
Did the smoothing/compression remove details? Did the multi-shot remove or add motion artifacts you wanted to see? Has noise-cancelling removed or altered frequencies? Is the high-frame rate real, interpolated, or anything inbetween depending on light just to make it look nice?
In the end, they're consumer devices. "Does it look good -> yes" is what thrums everything in this market. Expect the worst.
This has been true of consumer digital cameras for 25 years. It's not new to or exclusive to smartphone cameras. It's not even exclusive to consumer cameras as professional ones costing many times more also do a bunch of image processing before anything is committed to disk.
And they actually look good!
No phone can deliver that, even today.
Sony is probably the worst offender here actually. The a6x00 series applies lossy (!) compression to every RAW file without any option to disable, and there's an additional noise filter on long exposures that wreaks havoc on astrophotography:
https://stephenbayphotography.com/blog/sony-raw-compression-...
http://www.markshelley.co.uk/Astronomy/SonyA7S/sonystareater...
"You just took a picture of the Eiffel Tower. We searched our database and found 2.4 million public pictures taken from the same location and time of day. Here are 30,000 photos that are identical to yours, except better. Would you like to delete yours and use one of them instead?"
edit: for a while anyway
The current narrative in financial places seems to be 2023 is gone but 2024 comes with a vengeance, feel safe investing, your real estate will grow in value! We're making people go back to the office so that it does! More lemmings! Less quality! More production!
Yes. I'm peak cynical after 3 years of self made economic destruction only to tell those who didn't cause it that "of course they must pay for the broken plates".
Also, everytime someone invests a lot in explaining themselves in HN, if it goes against the current thread narrative, it'll still be ignored so it's not like it's particularly worth it. HN feels more Reddit and less Slashdot of old.
Still a nice aggregator to find stuff that might peak interest.
You got a friend, spouse or someone close that has hundreds of pictures of you on their phone. Their phone has a "AI chip" that is used to finetune the recognition models and photo models with your AI library. Like Google Photos tags images of people you know, so does the model. It also helps sharpen images - you moved your head in an image and it was a bit blurry, but the model just fixed it, because like the original model had for the moon, it has hundreds of pictures of you to compensate.
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
Once generative AI really takes off we will need some system for unambiguously proving where an image/video came from; the solution is quite obvious in this case and many have sketched it already.
In a world where any image or video can be generated, chain of custody to a real-world ground-truth will be vitally important.
This will become indispensable sooner than anyone imagines or wants it to.
Absent a chain of custody (perhaps including GPS baked into the signed image data blob), I think analog artifacts will become untrustable. Unless you can physically date them to pre-generative era!
Bonus feature: new display devices get better trained / new AI features for free
E.g. from the raw data we know the sensor, we can train a specific AI to enhance the image from the known physical properties of the CCD
Sometimes there are exploits which extract these keys from the cameras themselves, but I don't hear them nowadays.
One of the older products: https://imaging.nikon.com/lineup/software/img_auth/index.htm
Likewise depth fields and other potential forms of sensor augmentation.
For the long time digital cameras embedded in EXIF metadata about conditions on which the photo was made. Like camera model, focal length, exposure time etc
Nowadays this metadata should be extended with description of AI postprocessing operations.
Of course. But to ensure that's valid for multiple purposes we need a secure boot chain, and the infrastructure for it.
To get there we need an AI arms race. People trying to detect AI art with machine learning vs. increasing AI sophistication. Companies trying to discourage AI leaks of company secrets and reduce liability (and reduce the tragic cost of mistakes of course) vs. employees being human.
Or we could have built a responsible and reasonable government that can debate and implement that.
Maybe I'm naive. I'll take responsibility for that.
In the meantime, it's playtime for the AIs. Bring your fucking poo bags, theyre shitting everywhere (1), pack it in, pack it out.
(1) what the world didnt know, was that this was beautiful too.
Or we can just recognize the lunacy of it and opt out of caring. You can't stop the flood, so you just learn to live with it. With the right view, the flood becomes unimportant.
What measures can the government implement to combat this? AI image modification is realistically possible even on consumer hardware running locally. There is no going back.
Some image formats (e.g. HEIF) already allow to store multiple images in the same file.
This "future" is present in current Pixel lineup btw. Photos are tagged as unblured, so for now you can still safely take a selfie with your friends.
Photos taken by cell phone cameras increasingly can't be trusted as evidence of the state of something. Let's say you take a picture of a car that just hit a pedestrian and is driving away.
Pre-AI, your picture might be a bit blurry, but say, it's discernible that one of the headlights had a chunk taken out of it; it's only a few pixels, but there's obviously some damage, like a hole from a rock or a pellet gun. Police find a suspect, see the car, note damage to the headlight that looks very close, get a warrant for records from the suspect, find incriminating texts or whatnot, and boom, person goes to jail for killing someone (assuming this isn't the US, where people almost never go to jail for assault, manslaughter, or homicide with a car) because the judge or jury are shown photos from the scene, taken by detectives in the street of the person's driveway, and then from evidence techs nice and close-up.
Post-"AI" bullshit, the AI sees what looks like a car headlight, assumes the few-pixels damage is dust on the sensor/lens or noise, and "fixes" the image, removing it and turning it into a perfect-looking headlight.
Or, how about the inverse? A defense attorney can now argue that a cell phone camera photo can't be relied upon as evidence because of all the manipulation that goes on. That backpack in a photo someone takes as a mugger runs away? Maybe the phone's algorithm thought a glint of light was a logo and extrapolated it into the shape of a popular athletic brand's logo.
Yeah, but in the future the government will know your precise location, all day, every day, so at least you'll have an alibi.
AI isn't the thing to be worried about. People with power abusing AI is the thing to be worried about.
Sounds like pretty standard forensic science, like bite marks and fingerprints.
If any of you young folks haven't watched Ghost in the Shell, just close this tab and do that.
http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
So far as law and justice goes it is the other way around too. If it is known to be possible that cameras can hallucinate your identity, it won't be possible to use photographic proof to hold people to account.
This of course isn't always the case. When something is really important or significant people sometimes do want to know the truth as best they can. I want to know the car I'm purchasing isn't a lemon, I want to know the home I'm buying isn't a money pit, I want the doctor to to tell me if my health is good or bad (for some, under the condition the information is actionable), and so on.
When it comes to more frivolous things, for many, build the fantasy, sell them that farm to table meal you harvested from the dew drops this morning and hand cooked with the story of your suffering to Michelin star chef and how you're saving my local community by homing puppies from the local animal shelter with profits... even if you took something frozen, slapped it in the microwave and plated it and just donate $10 a month to your local animal shelter where you visited twice to create a pool of photos to market. For many, they want and crave the fantasy.
Progress made by science and tech has, for a brief fragment of history, established techniques and made practical, in some cases, to peel away all or at least some layers of fantasy away to reality. We started to pierce into cold hard reality and separate the signal of truth, as we can best understand it, from all the noise of ignorance and fantasy.
For many fantasy lovers, snakeoil salesmen, and con men, pulling away the veil of fantasy and noise has been a threat and there's been a consistent battle to undermine those efforts. The whole emergence and perpetuation of misinformation and recent "fake news" trends are just some of the latest popular approaches. We've been seeding our knowledge and information more recently with increasing degrees of falsehoods and pure fabrications.
Now, enter "AI," especially generative flavors. The same people who wanted to undermine truth are foaming at the mouth at the current ability to produce vast amounts of noise that in some cases are almost indistinguishable from reality from current techniques we have. Not only that, fantasy lovers en masse are excited at the new level of fantasy they can be sold. They really really don't care or want the truth. They really do just want "a good looking picture", "to make the summary interesting", or just see some neat picture. They don't care how accurate it is. Now people interested in the truth are facing a deluge of technologically enabled difficult to seperate noise production.
Is what I'm looking at close to reality? How many layers of noise are there I should consider when interpreting this piece of information? In the past, the layers used to be pretty managable, they were largely physical limitations or resource limitations to falsify the data to a point that couldn't be easily discerned. These days... it's becoming increasingly difficult to determine this and more and more information in various forms are leveraging more sophisticated and believable noise production. Technology has made this affordable to the masses and there are many parties with interest in setting the clock back to a world where the best story tellers are looked at as the oracles of modern time.
People often scoff at ChatGPT that it seeds or "hallucinates" to interpolate and extrapolate gaps of knowledge and make connections but it does so in a way that people like. It projects confidence, certainty, and in many cases it gives exactly what people want. To me, it's scary because it's providing a service the majority seem to want and creating an onslaught of noise that's more costly to debunk than it is to produce.
The future is BigCorp's AI will censure your photo when you try to take picture like on tiananmen square massacre.
I want a photo, not even sharpend.
Eventually people won’t care much for clarity and precision, that’s boring. The real problem is that everything that can be photographed will eventually have been photographed in all kinds of ways. What people really want is just pictures that look more awesome, in ways other people haven’t seen before.
So instead, raw photos will be little more than prompts that get fed to an AI that “reimagines” the image to be wildly different, using impossible or impractical perspectives, lighting, deleted crowds, anything you can imagine, even fantasy elements like massive planets in the sky or strange critters scurrying about.
And thus, cameras will be more like having your own personal painter in your pocket, painting impressions of places you visit, making them far more interesting than you remember and delighting your followers with unique content. Worlds of pure imagination.
There's an arms race between people adding nonexistent details to old films and people manufacturing televisions with dense enough pixels to render those microscopic fictions. Then they lay a filter over it all and everything becomes smooth gradients with perfectly sharp edges.
In the other hand, there will likely be a market for makeup mirrors that emphasize your flaws (both to help the user and to sell more product)
AI will accelerate this process exponentially and just like in The Matrix, most people will eventually prefer the simulation to reality itself.
<soapbox>confabulated</soapbox>
Of course they would have perfect skin and expertly applied eye-liner and lipstick as well.
A "smart" scanner tries to compress images.
Not sure I like that future
A combination of ai models trained on high resolution textures and objects, models of the actors, and training from every frame of the movie that cal use the textures and geometry from multiple angles and cameras to “reconstruct” lost detail.
I am begging people to find a new gotcha for AI other than "training it on data created by itself or other AI."
It's an obvious issue with obvious solutions. If it happens, it will be due to ignorance. It is not inevitable, and it shouldn't even be likely.
If they applied a perfect digital gaussian-blur, then that is reversible (except at the edges of the image, which are black in this case anyway). You still lose some detail due to rounding errors, but not nearly as much as you might expect.
A gaussian blur (and several other kinds of blur) are a convolution of the image with a specific blur function. A convolution is equivalent to simply multiplying pointwise the two functions in frequency space. As long as you know the blur function exactly, you can divide the final image by the gaussian function in frequency space and get the original image back (modulo rounding errors).
It is not totally inconceivable that the AI model could have learned to do this deconvolution with the Gaussian blur function, in order to recover more detail from the image.
(Just in case, original image https://imgur.com/PIAjVKp )
Those rounding errors are very important though. The Gaussian function goes to zero very quickly and dividing by small numbers is not a good idea.
If your deconvolving a noise free version of the original that also doesn't have any saturated pixels (in the black or white direction) then you can get the pretty close to the original back. I don't think this applies here because the OP is taking a picture of a screen that shows the blurred version, so we've got all kind of error sources. I think the OP is right: the camera is subbing in a known picture of the moon.
It would be interesting to see what happens with anisotropic blur for example, or with a picture of the moon with added fake details (words maybe?) and then blurred.
> To further drive home my point, I blurred the moon even further and clipped the highlights, which means the area which is above 216 in brightness gets clipped to pure white - there's no detail there, just a white blob - https://imgur.com/9XMgt06
> I zoomed in on the monitor showing that image and, guess what, again you see slapped on detail, even in the parts I explicitly clipped (made completely 100% white): https://imgur.com/9kichAp
Given how small the pure-white areas are, tbh I'm not sure I'd consider that as having "added detail". It has texture that matches the rest of the moon, but that's about as far as I'd be comfortable claiming... and that seems fine, basically an un-blurring artifact like you see in tons of sharpening algorithms.
I do think this "clip the data, look for impossible details" is a very good experiment and one that seems likely to bear fruit, since it's something cameras "expect" to encounter. I just don't think this instance is all that convincing.
---
And to be clear, I absolutely believe Samsung is faking it, and hiding behind marketing jargon. The outputs are pretty ridiculous. They may not be "photoshopping on a texture", but training an AI on moon pictures and asking it to add those details to images is... well, the same thing Photoshop has features for. It makes no difference - it's not maximizing the data available, it's injecting external data, and they deserve to be slammed for that.
Well yes, but he also downsampled the image to 170x170. As far as I know, downsampled information is strictly lost, and unrecoverable without an external information source (like an AI model trained with pictures of moon).
[1] https://upload.wikimedia.org/wikipedia/commons/thumb/2/2b/Lu...
Rather, reversing blur of any type is limited (a) by spatial decimation (a.k.a. down sampling, which is performed in the article), and (b) by noise/quantization floor, below which high frequency content has been pushed.
Another point of view is that there are infinitely many images that will produce the same result after blurring. Obviously, this makes the operation irreversible.
No, it's emphatically not. Perhaps you are thinking of displaying a spectrogram.
To produce an image from frequency-domain data, inverse DFT must be applied. Since (as @nyanpasu64 points out), the DFT of a real-valued image or kernel is conjugate-symmetric (and vice-versa), the result is again real-valued without loss of information. The phase information is not lost. If it were, the image would be a jumbled mess.
(Not that DFT+inverse DFT is necessary for Gaussian blur anyway -- you simply convolve with a truncated Gaussian kernel.)
> Another point of view is that there are infinitely many images that will produce the same result after blurring.
No, this is not true. I don't know why you think it is. This is only true of a brick wall filter, which Gaussian filter is not [1].
The SNR of high-spatial-frequency components is reduced for sure, which can lead to irrevocable information loss. But this is nothing to do with phase.
[1] https://en.wikipedia.org/wiki/Window_function#Gaussian_windo...
[1] https://en.wikipedia.org/wiki/Window_function#Gaussian_windo...
Actually any noise distribution is frequently reversible if you know the parameters and number of steps. This is in fact how diffusion models work (there's even work of Normalizing Flows removing realistic camera noise). It is just almost impossible to figure this out since there are many equivalent looking ways. But we need to be clear that there is a difference between reversibility and invertibility. A invertible process is bijective, or perfectly recreates the original setting. A reversible process can just work in both directions and isn't guaranteed to be invertible. (Invertible means reversible but reversible doesn't mean invertible)[0]
I bring this up because even more complicated versions of bluring could be argued as not "faked" but rather "enhanced." A better way to test Samsung faking the data is to mask out regions. If the phone fills in the gaps then it is definitely generating new data. This can still be fine if the regions are small, unless we also want to call bilinear interpolation "faked" but I don't think most people would. This is why it gets quite difficult to actually prove Samsung is faking the image. I don't have a Samsung phone to test this though.
So basically I'm with you, and even a slightly stronger version of this
> It is not totally inconceivable that the AI model could have learned to do this deconvolution with the Gaussian blur function, in order to recover more detail from the image.
Edit: After reading other comments I wanted to bring some things up.
- The down scaling is reversible, but not invertible. We can upscale, reversing the process. But yes, there is information lost. But some data can still be approximated and/or inferred.
- The clipping experiment isn't that good. Honestly, looking at the two my brain fills in the pieces and they look reasonable to me too. Clipping the brightness isn't enough, especially since it is a small portion of the actual distribution. I did this on both the full image and small image and both are difficult to distinguish by eye from the non-clipped. Clipping below 200 seems to better wash out the bottom of the moon and remove that detail. 180 seems better though tbh.
> But yes, there is information lost. But some data can still be approximated and/or inferred.
The perfect summary.
A smartphone sensor pixel has space for some low 4 digits number of electrons (created with some probability from photons, but that stochastic effect doesn't matter for anything a normal user would photograph) and typically should have a fixed 2~10 electron standard deviation from the analog-to-digital-converter (well, mostly the amplifiers involved in that process).
So if your pixel is fully exposed at a high 10000 electrons, and you √ that, you have 100 electrons stddev from shot noise plus worst case 10 electrons stddev from the readout amplifier/ADC. If you have a dark pixel that only got 100x less light to only have accumulated 100 electrons, √ of that gives 10 electrons stddev of shot noise plus the same 10 electrons stddev readout amplifier/ADC.
The problem is that while you have an SNR of 5 with the dark pixel, when trying to deconvolve it out of a nearby bright pixel, even perfectly with no rounding errors (1 electron = 1 ulp/lsb in a linear raw format), you now have 100/110 = 10/11 ≈ 0.91. That's far worse than the 5 from before. This gets worse if your ADC has only the 2 electrons stddev instead of the 10 (about 2x worse here).
That's the reason why deconvolution after the photon detector is a band aid that you only begrudgingly tend to accept.
The trade-off just requires massively increased aperture/light gathering, likely negating your savings on optics.
Not true. Deconvolution is a statistical estimate. Think about it. When you blur, colors get combined with their neighbors. Statistically this moves toward a middle grey. You're compressing the gamut of colors towards the middle, and thus losing information. Look at an extreme case - 2 pixels of mid-grey. It can be deconvoluted to itself, to a light and dark grey, or to one black and one white. All those deconvolutions are equally valid. There's no 1-to-1 inverse to a convolution. If you do a gaussian blur on a real photo and then a deconvolution algorithm you'll get a different image, with an arbitrary tuning, but probably biased towards max contrast in details and light noise, since that what people expect from such tools and what most real photos have. But, just like A.I. enhanced images, it's using statistics when filling in the missing data.
I wonder if there is some algorithmic way to find the key and tell if it's correct - some dictionary attack, or some loss function that knows if it's close. Perhaps such a thing only works on images that are similar to a training set. It wouldn't work on black and white random dots, since there'd be no way for a computer to grade or know statistics for which deconvolution looks right.
Given a perfect blurred image, reconstruction is possible - however due to the attenuation, these high frequency components are ~sensitive~.
Apart from quantisation effects [you mentioned which limits perfect de-convolution], adding a little AW Gaussian noise(such as taking a photo of the image from across the room) after the kernel is applied obliterates high frequency features.
Recovery when noise is low (plus known glyphs) is why you should not use Gaussian blur followed by print screen to redact documents. Inability to recover when there are artifacts and noise is [part of] why cameras cannot just set a fixed focus [at whatever distance] and deconvolve with the aperture [estimated width at each pixel] to deblur everything that was out of focus.
TLDR for readers, It is unlikely to recover sufficient detail via de-convolution here.
That being said, there were a few comments on here about gaussian blur and deconvolution, which I would like to tackle. First, I need to mention that I do not have an maths/engineering background. I am familiar with some concepts, as I've used deconvolution via FFT several years ago during my PhD, but while I am aware of the process, I don't know all the details. I certainly didn't know that the image that was gaussian blurred could be sharpened perfectly - I will have to look into that. In fact, I used gaussian blur to redact some private information (like in screenshots), and it's very helpful to know if I haven't redacted anything and the data is recoverable. Wow.
I would love to learn more about the types of blur that cannot be deconvoluted.
However, please have in mind that in my experiment:
1) I also downsampled the image to 170x170, which, as far as I know, is an information-destructive process
2) The camera doesn't have the access to my original gaussian blurred image, but that image + whatever blur and distortion was introduced when I was taking the photo from far away, (whatever algo they are using doesn't have access to the original blurred image to run a perfect deconvolution on)
3) Lastly, I also clipped the highlights in the last example, which is also destructive (non-reversible), and the AI hallucinated details there as well
So I am comfortable saying that it's not deconvolution which "unblurs" the image and sharpens the details, but what I said - an AI model trained on moon images that uses image matching and a neural network to fill in the data.
Thank you again for your engagement and your thoughtful comments, I really appreciate them, and have learned a lot just by reading them!
Absolutely never do that. I honestly don't understand why people still do, given that it's obvious that low levels of blur can be reversed why even risk guessing until what point someone might be able to recover anything? Just censor it, draw over it with an opaque tool, and save it in a format that won't store layers or undo history or something (the riskiest format being pdf).
If you don't like how that looks, the alternative is to replace the information and then blur it. They can unblur but will find an easter egg at best.
Personally, I censor instead of blurring a replacement, but I balance between low contrast and not hiding the fact that information was removed. A stark contrast distracts and looks ugly. E.g., for black text on a white background, I'd pick a light/medium gray (around the average black level of the original text, basically).
For eliminating such risk, just screenshot your censored content and use that image.
Does it just overdraw it (i.e. erase it), apply texture over the foreground element, or fail altogether?
This scenario is probably why other vendors don't go so far as to fake such images with texture overlays.
https://i.ibb.co/Kz7Sbm2/8-EA85-C12-5-B11-44-D8-9566-461-C98...
It's absolutely painful and encourages me yet again to use my mirrorless camera even more.
AI has been used in "cell phone photography" for a few years, at lease since Pixel 2 where a mediocre sensor produced much better pictures than what people expect (maybe there are other players who did this even earlier). And every manufacturer started doing it, including Apple. Otherwise, do you think "night mode" is just pure magic? Of course not, algorithms are used everywhere.
How do you define "fake"? In podcasts, Verge editor Nilay Patel has asked various people "what is a photo", because the concept of a "photo" has become increasingly blurry. That is the question the author is asking, and people may have different answers from the author's.
Curiously and revealingly, the political word photo-op stands alone in this photo- parade of words in the age of photo-imaginings. The universe does indeed have a sense of humor.
Night mode definitely uses some AI but most of the result is from stacking frames. Samsung here did not label it as a "scene optimizer". Their marketing just calls it Space Zoom. The only disclaimer they provide is "Space Zoom includes digital zoom, which may cause some image deterioration."
[ Overview of moon photography]
Since the Galaxy S10, AI technology has been applied to the camera so that users can take the best photos regardless of time and place.
To this end, we have developed the Scene Optimizer function, which helps AI to recognize the subject to be photographed and derive the optimal result.
From Galaxy S21, even when you take a picture of the moon, ai recognizes the target as the moon through learned data, and multi-frame synthesis and deep learning-based ai technology when shooting. The detail improvement engine function that makes the picture clearer has been applied.
Users who want photos as they are without AI technology can disable the optimum shooting function for each scene.
[1] https://r1-community-samsung-com.translate.goog/t5/camcyclop...The problem is that AI isn't just interpolating data. It is wholesale adding extra data that simply doesn't exist. The person in the background is facing left, but the sensor couldn't possibly have captured that detail even after multiple images--it was a coin flip that the AI made.
The issue is that, like privacy, most people won't care ... until they do. By that time, it will be too late.
Someone here included an example where it does do something like this: https://news.ycombinator.com/item?id=35109568
Is is ridiculous that OP consider this "cheating". Most people just want a nice picture and don't give a damn about AI.
nice.
I'm sure that they have started using AI to fill in details more recently, but this is just to point out clever use of multiple exposures and AI can help without faking detail.
It seems that Samsung simply adopted the same tactic to compete...
On the Huawei "Moon Mode" controversy, one can even find a research paper [3] published in a peer-reviewed social studies (!) journal, Media, Culture & Society:
> This is where the controversy began: Chinese tech critic Wang’s (2019) posting on Weibo, the Chinese equivalent of Twitter, made quite a splash. In his post, Wang put forward a shocking argument: he said that Huawei’s Moon Mode actually photoshops moon images. He contended that, based on his self-conducted experiments, the system ‘paints in pre-existing imagery’ onto photographed takes, re-constructing details that are not captured in the original shots. Huawei immediately refuted these claims, stressing that the Moon Mode system ‘operates on the same principle as other Master AI modes that recognize and optimize details within an image to help individuals take better photo'
[1] https://www.androidauthority.com/huawei-p30-pro-moon-mode-co...
[2] https://www.phonearena.com/news/Is-the-Moon-Mode-on-the-Huaw...
[3] https://journals.sagepub.com/doi/full/10.1177/01634437211064...
https://petapixel.com/2020/04/21/huawei-accidentally-claims-...
All the subtle trickery manipulation that the smart phone's doing to reality is concerning. Smoothing people's faces, making their eyes pop, enhancing the shit out of the colours, and now plopping fake objects overtop of the real ones.
Future concerns of this technology should range from a low-key disconnect from reality, to the complete inability to photograph certain objects or locations.
Imagine dusting off a 30 year old digital camera, finding some AA batteries to put in it, snap a selfie and then realizing just how ugly we all are and how washed out the polluted world actually looks without a bunch of narcissism-pandering enhancements.
This phrase could cast a broadening net with each year’s new tech.
Our brains do far worse stuff with our memories. Not sure how relevant 'high fidelity' is to people who mostly use phones for memories
Compare this to AI/ML-manipulated photos that aren't labelled as such. In this case we no longer know if it's our brain fudging a picture or our camera, so we are starting to lose an arbiter of truth.
Side-rant but it'll be weird when this makes it into high-end cameras that save RAW images. Will RAW still be "raw?" I don't know enough to say.
"The Hottest Gen Z Gadget Is a 20-Year-Old Digital Camera
Young people are opting for point-and-shoots and blurry photos."
https://www.nytimes.com/2023/01/07/technology/digital-camera...
I think it's great. It's the exact opposite of the person who spends all their time gear shopping and never using the gear.
The remaining budget can go into going to nice places to shoot nice photos, and to print them nicely and large enough to put on your wall choice.
I picked Nikon, but I assume Canon works as well.
The G9x Mark II, Sony RX100, Panasonic LUMIX and similar 1" sensor cameras are awesome though and I don't think they've gotten too crazy with computational photography. I imagine some color processing modes might be doing a bit of work though.
Strictly speaking, applying a Gaussian blur does not destroy the information. You can undo a Gaussian blur with a simple deconvolution, which is something I would expect even a non-AI image enhancement algorithm to do (given that, you know, lenses are involved here).
I'd like to see what detail can be "recovered" with just the downsizing, which DOES destroy information.
I photoshopped one moon next to another (to see if one moon would get the AI treatment, while another would not), and managed to coax the AI to do exactly that.
This is the image that I used, which contains 2 blurred moons: https://imgur.com/kMv1XAx
I replicated my original setup, shot the monitor from across the room, and got this: https://imgur.com/RSHAz1l
As you can see, one moon got the "AI enhancement", while the other one shows what was actually visible to the sensor - a blurry mess
I think this settles it.
I picture someone 20 years from now trying to find out what their parent really looked like when they were young. The obviously smoothed-out face filters are already giving way to AI-powered homogenization. And the filtering is moving deeper down the stack from the app to the camera itself. There will be no "original".
"Is the Galaxy S21 Ultra using AI to fake detailed Moon photos?": https://www.inverse.com/input/reviews/is-samsung-galaxy-s21-...
Note that the author decides that Samsung's photos are "not fake", in the sense that they were not doctored after being taken with the phone. However, the article decisively proves that they're being heavily doctored in-camera.
Another test would be to shoot RAW + JPEG if the camera supports it. A true RAW image would reveal what the sensor is actually capturing.
BUT, it also provides apps with true RAW, which is true (or very close to true) raw.
Two different definitions of "raw" is apparently possible even on the same device.
Thank you for that detail. IMO, that makes "ProRAW" a really unfortunate naming decision on Apple's part. And I think you're saying that if you shoot in RAW with, say, Halide, the result will be an actual RAW (pre-demosaic) file in DNG format.
A couple of interesting articles on the topic: (1) https://lux.camera/understanding-proraw/ (2) https://www.austinmann.com/trek/iphone-proraw
Sensor: 23.5 x 15.6mm 24MP
S21 Ultra: https://cdn.discordapp.com/attachments/1010562706237038633/1...
Telephoto sensor: 3.3 x 4.3mm 10MP (240mm equivalent)
Compare side by side: https://imgur.com/a/QwnV99D
This is to get an idea of quality, with the APS-C camera having a much larger (26x area) sensor and better optics to work with.
My impression is... S21 Ultra "space zoom" is, at best, a good party trick. But if you zoom in, the quality is still nearly garbage. Not an objectively "great photo of the moon."
It still shows you much more details than visible via eyes, so yeah its a party trick (what else would moon shot on phone be), but pretty darn great at that (I haven't seen so many people with :-O since iphone 1 release when showing this... then they quickly try their top iphones and xiaomis and end up consistently with a small white blob).
There is one aspect that this phone wins at easily - it can take that moonshot (TBH it can be a bit sharper than yours) while handheld, pretty consistently. Good luck trying that with your apsc with such a long shutter, it will consistently end up in just a blur. Software often beats raw hardware even these days.
https://cdn.discordapp.com/attachments/1010562706237038633/1...
I can clearly see that most folks here don't actually own discussed devices (which is fine, its US-based HN, a bastion of iphone and many Apple employees dwell here and uncritical appreciation of Apple is very evident in every single related thread). I've used its 10x zoom extensively over more than a year, it simply blows all other phones away easily for that kind of situation (more than those rather weak 3x zooms available everywhere). Family photos, wild animals, nature, anything you want to come closer, otherwise the scene is tiny dots in the center like on other phones. It works really well for what it is, with obvious unavoidable physical limits.
Overall this phone made me put my fullframe Nikon D750 away on a day I bought it. I took it 'just to be sure' on vacation to Egypt last year, didn't touch it a single time. Most often it doesn't produce strictly as good images but a) they are good enough to be viewed on phones side by side easily, basically as good as fullframe there and sometimes even much better, ie handheld photos in the night of dark scenes, fullframe is utterly lost without tripod, and b) it weights 0 and takes 0 extra space (and cost 0 instead of many thousands for modern camera with big sensor), since I have phone with me always anyway.
Tried exactly the steps as author of article, couldn't reproduce it a bit, tried various mega zooms, his various original photos, dark room etc. Blur remained blur, nothing added. I mean at this point everybody acknowledges any decent phone is painting quite a bit (ie iphone taking other side of bunny than reality, thats a fine example) and I am sure Samsung is doing their part as they have the literal android flagships.
It would be interesting to see this tried on a source image that isn't the moon. Just white with a few dark spots. Does it actually add in completely new craters, or just where there are existing smudges? Or do something like half of a moon photo and have white, does it add craters to the white side?
The OP tried to do this by changing the contrast but I failed to see any craters appearing where there wasn't already dark spots in the source photo.
It does seem strange that the OP is using an image of the moon to start and that they don't provide a still shot of the one where they modified the brightness levels to cause clipping. It doesn't really "drive the point home" as claimed.
Of course the answer to these may be that you need something moon-like enough to trigger the moon optimizations. But if that is the answer it would be interesting to see something that comes right up to the threshold where it either snaps in and out of these optimizations or two very similar images produce widely different results.
That was the point of the Gaussian blur. By blurring the source image of the moon before taking its picture, there was no information. The Gaussian blur destroys fine detail, by design.
The image enhancement applied by the Samsung phone is adding detail where there was none originally – not just detail that might be buried in the optics somewhere, but not there at all. It involves a computational model of what the target (here the moon) "should" look like and guesses at what it thinks are the blurry bits.
How long until your phone detects the subject based on geo localization and replaces your shot with a stock image of the subject from an immense database of selected, professional-looking pictures?
https://r1.community.samsung.com/t5/camcyclopedia/%EB%8B%AC-...
Ongoing arguments over what this counts as:
https://old.reddit.com/r/Android/comments/11or39c/samsungs_a...
> However, the lunar photography environment has physical limitations due to the long distance to the moon and lack of light, so the actual image output from the sensor at high magnification is noisy and not enough to provide the best image quality experience even after compositing multiple images.
> To overcome this problem, Galaxy Camera applies an AI detail enhancement engine (Detail Enhancement technology) based on deep learning at the final stage to effectively remove noise and maximize the details of the moon, resulting in bright and clear moon photos.
It's not just this one thing, though. It also describes moon detection resulting in: setting brightness, using optical stabilization, "motion sensor data and video analysis"¹ for stabilization (VDIS), and fixing the focus at infinity. The "AI" magic is then described as being a detail enhancer, which could mean anything from {generically decreasing blur and improving contrast} to {applying a model that was purposefully overfit on the moon} (most likely it's the latter). Either way it's fake, obviously, but why do all the effort and make compromises to get a good moon shot if you're going to replace it anyway?
> If you shoot the moon in the early evening, the sky around the moon will not be the color of the sky you see, but rather a black sky, which is caused by forcing the image brightness down to capture the moon clearly. [I think this translation should have read "picture brightness"; elsewhere it also says "screen brightness" so I suspect the Korean word for "picture" is ambiguous]
So you can't shoot anything near the moon, like if someone is holding the moon up with a hand or something, presumably that would be all black. It's apparently still relying on the sensor to get most of the way there and using ML for the last leg.
Imo the feature should spawn a warning on screen "Details of moon filled in by computer and may differ from reality" with buttons for [ok] to quickly dismiss as well as [don't show again]. Then you can't not know that your images are being faked and it's not disingenuous, while most people would still appreciate the better quality because it'll be/work fine in 99% of cases.
¹ summary explanation of VDIS found on https://r2.community.samsung.com/t5/CamCyclopedia/VDIS-Video...
So in the future, there are either cameras that can see what others have seen before, and those that can truly capture new, true, detail (true as in, without filling it with estimations)
What happens if you manually add a few additional artificial details to your copy of the image -- like a trap street in maps. Does the camera slip up and show flawless moon instead?
The Gaussian blur they applied is theoretically reversible in a continuous function and infinite precision. In a discrete function (like a image in the computer) and only a few dozens of bits it's not 100% reversible, but it can be partially undone and get a sharper image (that is not as sharp as the initial image).
But there still might be advanced machine learning models, rather than simple filters.
https://en.wikipedia.org/wiki/The_Work_of_Art_in_the_Age_of_...
A digital image is a representation/abstraction/map kind of thing. And then a little bit of mental shorthand inside your head makes a connection and declares the thing and the representation-of-the-thing to be one and the same.
But it isn't the same thing. Not by a million miles.
It's a socially-accepted mindfuck is what it is.
https://old.reddit.com/r/Android/comments/11nzrb0/samsung_sp...
This is a cool and useful tech, but it obviously needs a good marketing story to avoid looking creepy. To think that no one will notice just makes it worse.
Binning the image, or cropping it in Fourier space, would be a better test.
The information is absolutely not gone.
Does it state that it hallucinates craters that were not there in the original, or is it possible the filters simply did an FFT, adjusted the power spectrum to what we expect of a non-blurry picture, hence inverting the Gaussian blur?
EDIT: Note that a deblur of a smooth but "noisy" image can cause "simulation" or "hallucination" entirely without AI. Could be any number of things causing an output image like that (wavelet sharpening, power spectrum calibration, ...). Even if the information isn't recoverable as such, a photo of a Gaussian blur has an unnatural power spectrum that could easily "trick" conventional non-AI algorithms into doing such things.
Especially since the only thing I see in the output is "more detail" (i.e. simply a different power spectrum than the author expected..)
> I *downsized it to 170x170 pixels* and applied a gaussian blur, so that all the detail is GONE
Continue to read through paragraphs 4 and 5 of the "Conclusion" section.
The input to the algorithm will be an image with a power spectrum that isn't natural, . It would be very natural even without "AI" to attempt to "deblur" when faced with such a power spectrum.
A deblur of noise can cause "hallucinations" with much simpler reasons than AI beong involved.
Could it not e.g. be doing some wavelet transforms or FFTs and automatic power spectrum calibration?
Looking at the images, it will be fantastical if not a violation of information theory if the phone didn’t use prior information about how the moon looks to create the photo.
Source: spent way more months than I would have preferred calculating CRLB values on Gaussian blurred point source images.
Was it always like this? Were we, people always like this?
I don't think HN (or Reddit) posters are arguing that deconvolution was actually being used, or that it would be practical to do so. Instead, they're just nitpicking about the technical correctness of this particular statement - "the (ir)reversibility of a perfect digital gaussian blur" - even if it doesn't affect the conclusion of the experiment.
It's just the standard nerd-sniping as seen in all tech communities... If the debate on whether you can "reverse a photo of a gaussian blur taken by a camera from the monitor" became hot enough, eventually someone may even spend a weekend to code a prototype to show how it actually works better than most people's imagination, with the solely purpose to win an argument on the Internet.
That said, Reddit is probably the worst place for it. I rarely read comments anymore there because of how 'dumb' they are. Worse, I feel like Reddit is like some self trained botnet at times. Go on any post, say 'blood is thicker than water', and without fail someone is going to tell you the original is 'blood of covenant is thicker than water of the womb.' This is of course incorrect, and the only place I've ever seen it is...Reddit. So people posting ridiculous things learning from people posting ridiculous things.
It happens here though, too, with the main difference being someone will usually correct you, and not be downvoted for doing so...
I’ve taken pictures of the moon at 100x optical zoom, and if Samsung is really faking this they’re doing a truly awful job of it.
Second - there's nothing wrong with an AI 'enhancement' option. It seems useful! But you should admit what you are doing and have an off switch. Imagine if all of the studio masters for the past 20 years were actually in lossy mp3 because the hardware switched silently?
Don't let them tell you this unless they can pass an ABX test.
No one complains about image or video compression either if the quality is good enough - because it’s not perceptually noticeable vs uncompressed; people can do A/B testing to be sure.
I imagine that in most cases, the "AI" would work just fine at guessing things. But important edge cases exist, including the possibility of adversarial machine learning spoofing attacks to create false images. Imagine faking a crime scene this way, it would something straight from cyberpunk fiction.
[1] Learning to See in the Dark https://cchen156.github.io/SID.html
Starting from the top - bribery, embezzlement, illegal transactions, stock manipulation, perjury - all the way down to their "partners".
They've been caught multiple times lying in their products description.
I won't add links as there are too many, but a quick search for Samsung and the keywords I mentioned will bring many results.
(let the downvoting begin)
https://www.reddit.com/r/Android/comments/11onztx/uibreakpho...
Oh nooooo this phones photos come out much sharper than my peanut brain expected!!! Fraud!!!!!!!!!!
> I downsized it to 170x170 pixels and applied a gaussian blur, so that all the detail is GONE.
You can get sued if you publish someones real pictures without makeup and photo-shopping. AI beatified pictures have the same criteria.
That sounds really weird. Any further info?
It looks more like moon mode is assembling fake detail out of shot noise from a photo being taken in the dark indoors. That's enough to cancel out his blurring and all that.
Even if it is guided upsampling with a "this is a moon prior", so what? That's not "fake", it's constrained by the picture you took.
Not denying that Samsung or any other brand would fake things, but this is in no way any proof at all.
1. Use Photoshop to blur a photo of the moon, destroying detail
2. Use a Samsung camera to take a photo of the blurred photo of the moon
3. The camera somehow emits a crisp photo of the moon, including the detail that was destroyed in step 1
It seems like the camera AI detected 'this is a photo of the moon', and used its knowledge of what the moon looks like to add the detail back in. Where else could the camera have gotten the detail from?