Update: iPhone Camera app did not replace person's head with a leaf
twitter.com
twitter.com
What kind of thing do you mean by "this type of thing"? Why do you think it degrades bayesian priors?
Life is, among other things, a constant quest to better understand the world around us. For every "real discovery" there is a ton of noise. Most of that noise can be easily discarded, some look more convincing. This particular instance looked more convincing.
We know that modern imaging equipment is very complex. You shoot a picture and complicated processes happen which were optimised by thousands of engineers to produce a good looking photo. In the past we have seen that such complex pipelines can introduce plausible looking artefacts in a different context: https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
This is why it could be possible that maybe something similar happens with the camera app. People investigated, and it turned out that that is not the case.
For every breakthrough in our understanding there were thousands of perhaps and maybes. For every eureka moment, there is a bin full of superluminal neutrino experiments. This is normal and part of the process of reasoning about the world.
If your thinking processes get damaged by encountering as of yet unexplained phenomenon then your brain is broken and you should ask for a refund.
"There wasn't an error but there could have been" is true, but as a reason to support invalid data it's very wrong.
> If your thinking processes get damaged by
What.
They just said wrong data makes them (and others) more likely to make mistakes in the future. None of this weird strawman.
First of all, there is no wrong data here. Twitter user X said their iphone did a funny thing. Twitter user X attached a snippet of the image and said they think the iphone can overlay leafs in certain circumstances over faces. That is it. If your conclusion from that is that "iphones has AI magic which overlays leafs on faces for sure" that is your hasty conclusion and it is on you. The data is not wrong, your conclusion is wrong.
> None of this weird strawman.
:( sombremesa drawn a hasty conclusion. When they realised that it is unsupported they said (direct quote) "I need to be more careful about what I click on (and let into my head..."
Which basically says: I should be less curious because look here, I ended up believing the wrong thing.
The problem is not being curious, but drawing hasty conclusions and believing the wrong thing.
They should stay curious, let a lot of information in their head, but engage in critical thinking. Read the weird twitter thread purporting to show the weird bug and think: What if the twitter user is mistaken? Is it possible that the picture doesn't show what they think it shows? What other possible explanations there are? And so on and so on.
That strategy is much more widely applicable, more robust against even intentional deceptions and leads to ultimately better understanding of the world.
0. https://pbs.twimg.com/media/FH0N9HNWQAE0n9R?format=jpg&name=...
And frankly looking at the original picture again, focus is no excuse for that. Yes, there is a leaf, no, the AI whizbang still blurred it all to crap.
Maybe…
Or maybe the wind was just causing the leaves to move, resulting in some blurring.
https://twitter.com/mitchcohen/status/1476399735858683906?s=...
Yes, the leaf itself is "clear" but what threw people off was the blurry and dark halo surrounding that leaf which looks like manipulation artifacts.
When you use Photoshop tools like healing brush, clone/stamp tool, or A.I. Content Aware Fill, you often get strange visual artifacts like that. Here's a quick example of erasing a person that leaves behind blurry artifacts: https://www.youtube.com/watch?v=eC4TsTRTHiY&t=1m42s
A simple application of Occam’s razor could’ve saved a lot of furor
Well, that's the crux of the issue that you don't seem to see. For many observers, it does not look realistic. The unnaturalness of the pixels is what makes it plausible for Apple software to have a bug. If it looked 100% realistic to everyone, we wouldn't be in this thread discussing it.
E.g. Image saving algorithm with a software bug can cause pixels to be incorrectly "replaced" even though the programmers never intended it: https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
EDIT reply to: >, especially when the image so clearly depicts that.
You still seem to be missing why this discussion of AI error was even possible. In this very thread, there are commenters who still don't see "clear depictions" of a realistic foreground leaf even after being made aware of parallax demonstration in the update... because the blurriness keeps hijacking the brain to make it look like a fake artifact. Consider that the actual person who took the original photo didn't realize it was a real foreground leaf. (Or so he claims.)
In other words, saying "Am I crazy? I don't see why there's debate about the dress being blue or white when ... it is clearly and realistically BLUE" -- doesn't actually add to the discussion. Hey, it's great that the foreground leaf looks realistic to you. But your personal perception is not relevant to why others perceived it as an artifact.
I’m much more likely to interpret this thing as anti-capitalist sentiment being expressed in a ridiculous fashion than an actual good faith argument there could be a bug so rare but so profound that people’s faces are mistaken for the background and the AI’s post hoc justification is that there is a branch and leaf there so it’ll output that. It doesn’t even make sense what is being claimed here, your links don’t really change that
Edit: I see a leaf in the foreground, and the situation was a highly (digitally) zoomed picture with a tree in the foreground. This has nothing to do with color perception like you imply; the fact that a bunch of people - whose perception was previously influenced to believe in AI error - don’t think it shows the exact literal situation it correctly shows hardly makes it a relevant discussion
Edit 2: what I see might not be relevant, but the camera software showed the situation correctly, and you’re still arguing because the digital zoom produced artifacts that the entire branch / twig / leaf that was literally there might NOT have literally been there and could have been added by some magic unexplained AI replacement.
It doesn’t even make sense. There is no room for human perception here. If the software was rendering leaves instead of faces, it wouldn’t be a single guy finding that scenario, and especially not when the argument the leaves didn’t exist in the first place was based on compressed video. Comical stuff. This is literally conspiracy theory argumentation from you
You keep repeating "I". Again, the emphasis on "I" in your sentence is not relevant to the commenter right above you perceiving something else: "The blurry plus the tiny twigs connected to those leaves still make my brain think it's some GAN artifacts."
The blue/white dress example was not to apply color to the iPhone example but a cultural reference to this perception debate: https://www.google.com/search?q=was+dress+blue+or+white
Or put another way, consider : https://en.wikipedia.org/wiki/Theory_of_mind
EDIT reply to: >This is honestly blowing my mind that you think theory of mind applies
Let me try to put it this way... nawgz sees a clear and realistic leaf in the foreground. Case closed. Therefore, nawgz would not even submit a Twitter post or a new thread to HN wonder why the head is missing. And yet, such a thread exists with (some) commenters wondering if the software had a glitch: https://news.ycombinator.com/item?id=29739235
Stepping outside of yourself and considering Theory Of Mind might help explain to you why such a thread exists with comments even though you yourself see no issue with the photo.
EDIT2: >There is no room for human perception here.
There is room for human perception because the person who took the original iPhone photo and posted it on Twitter is a senior iOS programmer with a degree in computer science and yet he overlooked the possibility of a real leaf. If we take his story at face value, his particular mind and perception initially chose to believe that iOS photo software replaced the head with a leaf rather than consider a scenario that a tree branch in front of his own house occluded the walker's head. Only when another programmer on Twitter theorized that a real leaf is in the foreground, he then went back outside again to notice that there's a low hanging branch on his tree. (That's his later video showing parallax demonstration.) The original photographer did not believe the pixels in his own photo. That's a human perception issue.
That faulty perception led to software theories from some programmers in that Twitter thread including this comment:
>When you open iPhone camera, it keeps recording. When you take a shot, it takes some frames from before and after you pressed the shutter and then merges those to create a detailed photo. One of this frame had this leaf on her head. -- from https://twitter.com/masnun/status/1476457354128281600?s=20
Edit: I explained how theory of mind applies - a bunch of anti-capitalist or anti-Apple folks who think that magical AI is everywhere broke this photo spectacularly. I don’t really care why people think it’s something that it’s clearly not; people think covid vaccines are meant to shorten lifespans, or that climate change is a hoax, or that god exists.
None of the justifications posted are even close to relevant or believable at a glance, let alone after seeing the reality of the thing
What this isn’t is a software issue, or something that was reasonable to point to as one; after all, the software was exactly correct, and to an unbiased observer in an immediately obvious way.
That halo is pretty clearly what's left of the subject's head after the phone has tried to get rid of it. It's difficult for me to understand how this wouldn't qualify as "replacing someone's head with a leaf". Sure, the leaf is there. But the head used to be there too, and in the photo it's been removed.
Clearly, what you say is in some sense true; digital zoom and post processing have left some artifacts. To me it sort of looks like there’s another leaf on the same branch as the first.
But when you are digitally zooming a tiny iPhone camera to photograph someone 50-100ft away and there’s multiple depths captured, it seems hard to imagine how the photograph would be crystal clear in that circumstance, and even further it’s crystal clear the post processing only failed to add clarity, and it certainly did not introduce a nonexistent leaf / twig / branch connected realistically as some software bug.
It looks like two images—one with the face obscured, one with the face visible—were combined. Notice that both the leaf and the person are sharply in focus, and there’s a distinct circular, blurry outline around the face where a transition occurs between the two images.
No, the leaf in the photo isn’t fabricated by iOS. No, iOS didn’t copy a leaf from elsewhere in the image to continue the pattern. But the conclusion, “iPhone Camera app did not replace person’s head with a leaf” is probably incorrect: it looks to me like it did, in fact, replace the person’s head with a leaf. It just so happens that the leaf was really there from the perspective of one lens or photo.
Wasn’t this documented as intended behavior on newer iPhones? Images from different lenses will be combined and faces replaced with clearer versions in some scenarios. In this particular instance, the “clearer” version was a leaf, as there was a leaf in the way.
This isn’t really bad thing; it’s an understandable bug in a piece of technology that usually works amazingly well. However, it does mean there’s room for improvement.
Those two things are exactly what everyone was assuming happened. So while you're right about what is going on with the camera behavior, I think you're disagreeing about a completely separate conclusion than everyone else arrived at.
> It just so happens that the leaf was really there from the perspective of one lens or photo.
In the original thread the photographer was fairly certain there weren't leaves in front, which is what started all of this.
> Those two things are exactly what everyone was assuming happened.
What? Look at the other discussion thread:
> The cause of this is image-stacking.
> The phone takes ~20 frames, over 0.2 seconds. In that time, lots of people and things in the frame move.
> Optical flow is used to track all moving parts of the image, and then 'undo' any movement, aligning all parts of the image.
> Then the frames are combined, usually by, for each pixel, taking something like the median or throwing out outliers and using the average.
> When the optical flow fails to track an object in more than half the frames, the 'outliers' that are thrown out can in fact be the image content you wanted.
> It happens with leaves a lot because they can flutter fast from one frame to the next, so tracking each individual leaf is hard. A few bad tracking results on more than half the frames, and all you end up seeing is leaves where there should be a face..
( https://news.ycombinator.com/item?id=29742539 )
There was nobody arguing that the phone hallucinated the leaf. The problem is that the phone removed the person's head so that it wouldn't pull focus from the leaf.
The general consensus was that yes, it was image stacking, but perhaps it was moving a leaf that was in the background over her face. That turned out not to be the case after the updated tweet mentioned there was a tree in the foreground.
As a fairly trivial example, years ago now I wrote a little article observing that a Sony ad boasting about how responsive their camera was (at a time when many digital cameras had a distinct delay) was using a rather famous wildlife photo shot on film. I also noticed that it wasn't quite the famous frame though it was still an excellent photo. Odd. Maybe something about rights.
You probably see where this is going. Subsequently I saw other ads in the series which had clearly mistimed versions of other famous photos. At which point the thing I had never considered clicked--to my somewhat embarrassment.
(In my defense, the wildlife photo used was certainly not an obvious example of bad timing unless you were familiar with the original and even then it was subtle.)
The people who made such comments were never replied to, and while I can't see upvotes my suspicion given how far down you have to scroll to read them is that no one cared to give those possibilities much credibility.
Anyways, the image I was commenting is the one clicking on the thread title takes you to. The head is definitely missing:
https://twitter.com/mitchcohen/status/1476951534160257026
What image do you see the head as being "clearly visible"?
I've seen some strange things from stitching photos for panoramic, it isn't a huge leap of logic to think some compression or interpolation algorithm went wrong.
Glad to know it was a problem of perspective.
Don't blame systems for bad input; garbage in, garbage out.
IMO this is also a good example of engineers looking for complex bugs and solutions which are simply not there.
I thought maybe it was just some processing glitch, maybe it was computational photography messing up, maybe it was just a question of focus, or maybe a mix of things.
This is pretty dumb, though.
In meatspace, his eyes are at a different vantage point focusing on the face, while cellphone cameras try to keep everything visible as sharp as possible.
This is one of the few times I'll ever feel sorry for Apple! I can only imagine the calls they're going to get from Karen's who didn't take good pictures to begin with and blame Apple for it because they saw an internet that one time that somebody's iPhone replaced a head with some leaves.
That post was already being widely used to justify the fairly ludite views on image upscaling by the judge in the Rittenhouse trial for example.
https://www.theverge.com/2021/11/12/22778801/kyle-rittenhous...
That's throwing his own incorrect mistrust of the technology into things. It wasn't based on precedent or founded by rational examples.
He also didn't provide alternate means for video playback that he would trust. It's reductionist.
At the very least, a competent prosecution should have then pulled up content in VLC and said "the source code for zooming is here and verifiable"
Dangerous precedent works both ways, I don't know which effect will be stronger.
Post now, think later
I think a lot of the HN crowd are susceptible to that because they know just enough to know it's possible but are not perhaps familiar with a given domain (I certainly know that applies to me).
It's perhaps some spin-off of the dunning-kruger observation... There must be a spot on the curve where a person knows just enough to be freaked out by what they're seeing, but not enough to explain it fully. Whereas here the Halide devs know the domain well enough to quickly diagnose it.
Absolutely think this describes what happened
I'd have probably been more skeptical myself had I not seen an ad earlier in the day of an Android phone removing people from the background of an image as easily as you'd crop it. Whoops!
there is that false story that gathered tons of likes and comments, even though a buried comment debunked the whole thing. the worst part is that it's since been shared in many Quora spaces. where it is gathering more likes and comments.
i stopped for a minute and wondered if the situation could have been prevented.
Though the aggressive denoising and small lens probably doesn't help.
I think it's mostly this. The photograph is incredibly noisy, very difficult to tell what's going on.
(btw, is the "Karen" reference really necessary? It's a sexist, racist insult that detracts from an otherwise interesting comment)
Karen is a pejorative term for a white woman perceived as entitled or demanding beyond the scope of what is normal. The term is a meme depicting white women who use their privilege to demand their own way.
Every culture, color, creed, nationality, and gender has Karens.
To anyone I offended, I sincerely apologize.
The truth of the matter is that I'll call anybody a Karen who is dumb enough to deserve the label. I don't care about color they are I don't care where they're from I don't care what they look like or how their hairstyle is done or what they're wearing or how many kids they have or how rich they are or how poor they are. I could have just as easily said a***e.
But I chose Karen because people who act like a Karen are idiots.
And also misused quite frequently. Its origin was a white woman who called the police on a group of black people who were doing nothing wrong other than being black in the wrong place.
Also, central park karen is a recent addition to the long history of using karen as a punchline, anytime you think you know the origin story, check know your meme first, you might be surprised: https://knowyourmeme.com/memes/karen
I don't think there's anything wrong with a little stereotyping: the only reason it's "bad" is that actual bigots ruin the fun, so to speak. If anything, I feel bad for people named Karen.
My daughter saw this in person, where a woman was mad at people at the butcher section of the grocery store, calling woman who bought a whole chicken an 'animal murderer'. Plot twist: the very same woman had chicken breasts and thighs in her cart, but because they were not a whole chicken, she disassociated them from the animal they came from. I rest my case.
Now it's been abused and overused to where I don't even know what it means anymore. Lady comes outside to ask TikTokers to quit doing burnouts in her yard and it's 'shutup kawen, stupid kawen LOL amirite'
I don't believe this was said jokingly, but just to be sure, this definitely happens all the time.
> btw, is the "Karen" reference really necessary
No, it wasn't necessary but at this point, what isn't considered a slur anymore?
It's not, "he took one photo and it came out wrong", it's, "humans everywhere took billions upon billions of photos and you only were told about the weirdest ones which fit a Narrative".
They're a trillion dollar company who don't give even the slightest little fuck about you. Stop feeling sorry for them, that is an insanely unhealthy way of thinking.
The actual explanation will get lost in the discourse and leaf-head will live on in the minds of the impulse-minded masses.
See also: Yanni/Laurel and the dress thing.
Well now you’ve just gone and given this thing a catchy name. That’s not exactly going to help matters.
#leafhead
Just look at the picture: https://twitter.com/mitchcohen/status/1476351601862483968/ph...
Imagine what a person's head would look like in relation to the size of the body. Where has the head gone? What's the nebulous dark stuff textured like leaves where the person's hair should be?
I am happy that the iPhone didn't invent the leaf, but it has pretty obviously done something weird with it.