How iOS 13 redraws your eyes so you're looking at the camera
twitter.com
twitter.com
Good old traditional audio-only phone conversations allowed you to presume that the person on the other end was paying complete attention to you while also permitting you not to have to pay anything even close to complete attention to her...Video telephony rendered the fantasy insupportable.
The proposed solution to what the telecommunications industry’s psychological consultants termed Video-Physiognomic Dysphoria (or VPD) was, of course, the advent of High-Definition Masking;
Mask-wise, the initial option of High-Definition Photographic Imaging — i.e. taking the most flattering elements of a variety of flattering multi-angle photos of a given phone-consumer and — thanks to existing image-configuration equipment already pioneered by the cosmetics and law-enforcement industries — combining them into a wildly attractive high-def broadcastable composite of a face wearing an earnest, slightly overintense expression of complete attention — was quickly supplanted by the more inexpensive and byte-economical option of (using the exact same cosmetic-and-FBI software) actually casting the enhanced facial image in a form-fitting polybutylene-resin mask,
Sad state of the world where the default in a business related App is essentially telling you that this is necessary. Can’t be healthy for society
I'd rather a client/vendor/coworker not be distracted if someone on the call has big nasty pimples, and I'd rather have a coworker with big nasty pimples feel more comfortable getting on a video call.
I think this is different than touching up models because we never end up seeing their true form. Though I remember when the fappening happened a lot of people mentioned how they felt better about themselves after seeing these people's true selves.
But it being made default and thereby de facto saying that everyone needs it is absurd. There is no way that something like this is not damaging to society in the long run
Perhaps the feature should just be renamed/rebranded to something like "smartly reduce image details to save bandwidth"?
Don't worry, we're sad in our own way.
But yes, it crosses the line where it's no longer a camera (as a feature, it does not capture reality)
When the person on the other end of the call is looking at the image of me on their screen, the physical truth is that they're looking away from the camera, but the emotional truth is that their eyes are looking at me.
When I'm chatting with loved ones, which truth is more important? Physical truth, or emotional truth?
Consider for a moment that an offset front-facing camera might be just as serious a distortion of truth.
I don't think an offset camera is distorting anything. It captures the image it's expected to capture.
I think we've got a fundamental disagreement here. We'll just have to agree to disagree :-)
Reading these conversations i'm going a bit crazy as you try to rationalize why this eyeball redrawing business is a good and necessary tech. I think it's complete trash. Is this the tech that makes apple worth billions of dollars?
Not trying to stir shit up with you. just generally curious as I have an extremely opposite viewpoint. I am 28, prime millenial love to use apps and phone age. I am a huge nerd and geek (clearly as we are commenting on HN lol). I have literally never face-timed anyone except when other people are doing it and I am nearby.
I’m now 38 and I’ve seen my two year old son’s face light up when he sees his mum through the phone screen.
My wife’s parents are on the other side of the planet and their intercontinental video chats connect them to our family.
I love science, math, philosophy and photography. I should care about pixel perfect truth. But to me the value of FaceTime is entirely emotional. I say that when my wife looks into my eyes, I want to see her eyes looking at mine. The fact that this isn’t how it works now is because of optical distortion. While this isn’t a true holistic fix for the underlying distortion—that would be amazing—it looks like a neat solution.
I’ve not seen it live and I’ll reserve final judgment till then. If it’s even slightly noticeable I’ll absolutely hate it. But if I can see no sign of trickery when staring into my wife’s eyes, it might just make the world a happier place, even if only a tiny bit.
After relocating for college and again for work, I have family across four timezones and friends even farther abroad. In my experience, nothing short of a flight bridges that distance better than video calling. I can see my mother's eyes light up when we haven't spoken in a while, my younger brother can hold up his latest pair of sneakers to the camera while he geeks out about them, and watching my best friend's new puppy bounce around his apartment is much more interesting than hearing it bark in the background.
I'll probably make a FaceTime call 3-4 times a week, which from observation and casual conversation appears to be a similar frequency to the rest of my friend group (mostly coastal millennials, largely Bay Area + NYC, ages 24-31).
With regards to the eye repositioning I agree with the grandparent poster. All captured images are reproductions of reality, and if I were to place this one on a spectrum between corrective glass elements for lens distortion and cat-person filters I would say it's much closer to the former. In my opinion the correction captures the intent of the user, and I feel that's truthful enough.
This is more akin to doing a white balance adjustment, such that you see what the scene is expected to look like rather than the raw data captured by the camera, it's just correcting for camera location instead of exposure.
Another way to think about it is this is a software fix for a hardware limitation. In an ideal world we'd fix the hardware limitation, remove the software fix, and your experience would be unchanged (or actually, even better, because of the lack of distortion on stuff crossing your eyes).
“Emotional truth” is the wrong phrase. It’s the actual truth—the person on the other side is looking at your eyes.
The real issue is that the processing isn’t sophisticated enough. Instead of just moving the eyes, it should calculate what the camera would show if it were located in the screen.
But yeah, I think it's creepy too.
On an artistic level: https://toinformistoinfluence.files.wordpress.com/2012/06/19...
Digital cameras don't "capture reality", either. They scan the image over a small period of time, same as your average display in reverse. It's essentially a really fast time lapse, on the pixel level. Arguably, that's as much "not reality" as what Apple is doing here.
I mean, I guess I understand the idea of "capturing reality as close as possible". I just don't see much merit in caring too much outside of an academic context. This isn't deepfakery. This is shifting eyes to provide the much needed feedback of eye contact while on a video call.
I suppose that goes without saying, seeing as that is sorta what cameras are (well.. many cameras. The central examples of cameras. IR cameras don’t quite have this property, but you know what I mean.), but sometimes it is “necessary to state tautologies” (though this isn’t a tautology exactly).
A linear representation of a group needn’t distinguish between every pair of group elements for me to say that it “captures” some aspect of the group.
CCDs are just stupid light buckets and the reality of them is, at best, a spreadsheet showing those counts. Everything beyond that is a choice someone made to present things a certain way.
And everything before the photograph is taken is that way too. Framing, exposure time, time of day, choice of subject... etc. all influence the “reality” captured in the photograph.
Half the reason you probably think they capture reality in the sense you mean is because photographers not shooting for Instagram generally try very hard to make it appear that way. But... they don’t.
I think you're making 2 mistakes:
- first, thinking that the line is between what we think of as normal camera usage, and eye correction & deep fakes. But the image that we see now is already a result of image correction—white balance, exposure, lens adjustments, and and more secret processing that we don't even get told about.
- second, that it's a hard line. The possibilities aren't "no adjustment whatsoever" and "deepfakes". It's a gradient; there are minor adjustments (like white balance), moderate adjustments (like eye correction), and even greater adjustments (like deep fakes). It's not a hard line line at all. Eye correction is much closer to reality than deep fakes, and a lack of eye correction is closer to reality still.
No, even analog photography, choices in developing certainly effect how the "light coming in" turns into a picture. As well as the particular camera used, and it's settings (aperture, ISO, shutter speed), and the film chosen.
The choices the digital photography system makes are no less choices with effects on the "captured light" than the ones made after you get bits out of the camera. They're just choices made by the engineers who designed the digital camera. There is no single 'true' way to "capture light".
> When a photographer chooses a particular brand or type of film, he’s already deciding what kind of reality he wants to show. Different types of film show reality and colors differently. Photographers pick film based on what they want the result to look like. Film photography never ever showed reality, not even close.
https://www.theworldaheadofus.com/blog/photography-does-not-...
> But despite the presumption of veracity that gives all photographs authority, interest, seductiveness, the work that photographers do is no generic exception to the usually shady commerce between art and truth. Even when photographers are most concerned with mirroring reality, they are still haunted by tacit imperatives of taste and conscience… Although there is a sense in which the camera does indeed capture reality, not just interpret it, photographs are as much an interpretation of the world as paintings and drawings are. Those occasions when the taking of photographs is relatively undiscriminating, promiscuous, or self-effacing do not lessen the didacticism of the whole enterprise. This very passivity -- and ubiquity -- of the photographic record is photography's "message," its aggression.
http://www.susansontag.com/SusanSontag/books/onPhotographyEx...
But I don't actually disagree that there is something new going on lately with new kinds of digital processing. I don't think it's "all the same" exactly, we definitely have new techniques of altering and manipulating photos that give us SO MUCH more power than we had before, that I think it's reasonable to say it's a new thing.
But the original photographs never "captured reality" without intervention or manipulation. A photo is necessarily an intervention and a manipulation, in a variety of different ways. A photo never truly "captured reality" in an objective sense, it always depended on the photographer's choices, and the details of the technology used, in ways that mattered for what was perceived in the photo. But yeah, we have whole new orders of magnitude of it available, I'd agree.
2. Workaround: it emits a flash which creates an undesirable side effect
3. Fix: we clean up in software
Is like?
1. Engineering constraint: The camera can't be positioned behind the LCD panel the user is actually looking at
2. Workaround: it's placed 1 inch above which creates an undesirable side effect
3. Fix: we clean it up in software
---
In both cases we get closer to where we'd be if the engineering constraint did not exist.
This is incredibly funny.
The solution is a little creepy I'll admit. But the fact that we are becoming increasingly accustomed to talking to people who appear to be staring at our necks is even creepier, in my opinion.
I propose they put that hole not in a corner, but in the middle of the screen, and position the remote person's face so that the pupil of one of their eyes is positioned exactly over the camera. That should give you the most natural interaction without distorting anything.
More seriously, I'm wondering if at some point the front-facing camera may be an intrinsic part of the screen. I've read that on-screen fingerprint sensors already use a kind of camera in the screen. If that were to become good enough to see images at a distance, that might actually work.
https://arstechnica.com/gadgets/2019/06/get-ready-for-under-...
That coupled with a lack of even controlled lighting, best case sample images probably tells a lot about the image quality.
I personally think it's too far, we should stick to bezels so we can have great hardware for cameras, speakers and screens.
https://www.theverge.com/circuitbreaker/2019/6/5/18654365/xi...
Like the girl who was doing a video facetime call on her cell phone all the way from waiting at the gate , boarding, sitting on plane. Call me an old grumper if you want, downvote if you want. but damn , da fugg is the deal get off my lawn with your video chats kids.
There have been a few comments now that mention people looking at necks instead of eyes as being creepy.
Now I wonder if anyone finds it creepy to talk to me in person, because I usually look at the speaker's mouth, not their eyes, at least when they are speaking. (I'm not actually sure where I look when I'm doing the speaking).
Looking at the mouth allows better error correction if the conversation is competing with noise.
People do notice direct eye contact though. Maybe someone could explain this. I'm kinda curious why directly in the eye we notice but there's more wiggle room around the face.
Edit: Just saw where someone else posted there's a toggle.
You can elaborate on what you think will happen and why, if this gains traction. Anything else is just... literally FUD.
To make sure I don't get banned for being ironic, that's intended light-heartedly to say lots of processing already goes on. The tool is attempting to represent what both sender and recipient usually intend.
I don't buy the slippery slope argument here, yet. All they're doing is shifting eyes and nose to compensate for physical offset from the camera. Seems like Animoji would be more creepy, as it's capturing your entire face movement.
That's how the slippery slopes get you. They look all gentle and inviting at the beginning like you can totally stand on them no problem, then the slippery gets you and you're sliding right on down.
There's no going back and we have no idea where we're headed.
That's the problem.
And It would have been creepy if this was coming from anyone else. But knowing this would only be available on FaceTime I don't see what's wrong with it.
When I click it I just get a sign in page.
I'm really wondering if/how iOS avoids this, or if it took a ton of extra work.
I'm certainly not defending this trend and I think it's incredibly unhealthy — especially for the average teenager who's already naturally self-conscious about their appearances. But a minor eye correction will be peanuts in the eyes of this crowd (no pun intended) compared to the amount of processing that most of their Instagram and Snapchat photos go through before being uploaded.
[0] https://www.theguardian.com/media/2018/mar/09/facetune-photo...
[1] https://abcnews.go.com/GMA/Living/photo-retouching-apps-affe...
Probably we needed deep learning to do this well.
[0] https://cgl.ethz.ch/publications/papers/paperKus12.php [1] https://www.catch-eye.com/index.html
This new iOS video chat software modifies your video stream to correct for this camera/screen offset by _redrawing your eyes_. The end effect is that the person you're chatting has the impression that you're looking into their eyes rather than at their neck.
The effect is subtle, but can be made more obvious by putting a straight wire in front of your eyes. With the software feature disabled, the wire appears straight. With the software enabled, the wire appears bent.
I told her, but she answered that she was not really staring at the camera. I guess it makes sense now.
:)