Eye Contact in Video Conferencing (2018) | Hacker News Reader

Eye Contact in Video Conferencing (2018)

itotd.com

65 pointswalterbell68 comments

Latency is the issue for me more so than eye contact.

On video chat sometimes I’ll say something like “mmhmm” or “yes” in agreement, and accidentally throw the speaking party off because my comment was delivered after a normal pause in speech due to latency.

This interruption of normal conversation flow is more problematic to me than eye contact. We both know were speaking into machines after all.

For some reason video calls have just never worked well for me. Every single time it's either latency, massive amounts of echo making it hard to speak, and "can you hear me" every minute or two. For some reason, it's 2019 and we as a human race still haven't figured out how to get a stable 480p video stream over gigabits of bandwidth.

Video conferencing worked perfectly when it was still done over circuit-switched digital telco networks. It’s only the Internet that has botched things up.

If by "botched things up" you mean "Made affordable." Nobody other than Vint Cerf will argue that circuit-switched networks are not superior from a QoS point of view, but they can be orders of magnitude more expensive to build and maintain.

Yes you now have a choice: you can have terrible video meetings over Zoom or whatever, or you can get a T-1 and some Tandbergs and have good ones.

They should be feasible to implement within a building, between buildings on the same street, and probably between geographically diverse sites of multi billion dollar companies that spend 30%+ of their labor hours on site-spanning meetings.

Maybe it's a good use for deepfake technology. Send the audio quickly (that works fine on a telephone, must be OK), then the receiver fakes up a talking head based on that.

You'd be surprised - audio is very slow. Telephone works because we are trained to compensate and also because there are no visual cues. (Weird thing - the old analog lines were much faster than our cell phones of today.)

I can recall using the Hack to force international calls over cable instead of satellite (to reduce latency) when I used to do a lot of international liaison for BT.

This will definitely never happen with “monitor”-style technology, because AR/VR with face tracking will do it. That’s probably five years out.

As a side note, I believe eye contact and 3D projected body language tracking are the limiting technology for virtual offices to leapfrog physical offices.

It’s my expectation that when those two technologies are cheap ($600 all-in-one device) it will massively disrupt both commercial real estate and any residential markets that are commute dominated.

I believe particularly in the Bay Area this will be the source of the next housing market collapse. Particularly since Facebook is heavily invested in telepresence now, and has a clear path to a device like I described within ~5 years.

Google and Apple, in all likelihood, will put out a similar device in that timeframe as well.

Do you have any links where we explore such prototypes?

You can look up some of the foveated rendering prototypes. The non-static ones have eye tracking.

I have seen a few headsets with downward facing cameras but not many. There are hard problems around both technologies so they are mostly research prototypes at this point I think, not consumer alphas.

Also I think some Leap Motion prototypes might be facing downward and tracking body. I seem to remember one that had the Leap sensor on little arms sticking out on little arms.

As I said I believe Facebook, Google, and Apple all have this tech on the far side of their roadmaps. So I don’t see all three failing.

But even if they do, the Leap Motion roadmap points there, and... while Leap is obviously mismanaged, new parent company is UltraHaptics. Hopefully they can get Leap into a headset....

Regardless, the Leap founder (Ed: David Holz) knows exactly what’s necessary for telepresence in VR, and I don’t think any amount of mismanagement will stop him from executing it EVENTUALLY.

But he’ll take 10 years, and Facebook might do it in 3. Apple could roll it out next year for all we know. They’ve prototyped all the tech already in ARKit.

David if you’re in this thread and you want to build a leap headset email me. We can just disassemble Oculus Go’s and add a Leap and... well you wrote the 6DOF code already right? Solder a coprocessor on the board that can do it without any battery drain. Maybe we have to design one, IDK. Sacrifice a kitten. Hire a stats PhD and make it “stochastic”. Put a neural net processor between the cameras and the computer. Who knows.

Then ship it with a Nintendo 64 emulator ONLY, that’s the entire OS, no games you have to sideload them. And mark it up to $999. Fund the company on preorders.

Strip off the Go’s plastic and CNC a custom enclosure, faceplate carved to the buyer’s design. Use a portion of every sale to fund artists creating faceplate designs for customers to choose from.

Needs IPD adjust, off ear audio

I work from home already. No way im getting a ‘downward facing camera’. I’ll have to get the Genie Lamp plugin and just float around the virtual office sicut spiritus.

Seems reasonable that you could choose to be disembodied.

You would still benefit from having a point location in the meeting room. That’s one of the basic improvements VR gives you over videoconferencing. Maybe two floating eyes (for gaze direction and eye contact) plus some canned emoji-style gestures?

My point is you, as a person for whom current gen VR is enough—no eye or body tracking needed—are atypical. Most workers need body language and eye contact to communicate effectively.

Microsoft have also been researching this area for >15 years. https://www.microsoft.com/en-us/research/project/eye-gaze-co...

> Eye contact is extremely important for meaningful communication

Some of us, specifically blind people, have lived our lives without ever being able to make eye contact, yet we still have meaningful communication, both with each other and with sighted people. I, for one, don't ever think about my inability to make eye contact, except when a sighted person brings it up (edit: and to be clear, that only happens in a discussion like this one, not when trying to communicate with me personally).

i suspect eye contact is like most things in our complicated world, which is to say it exists among multitudes of culture, accessibility and familiarity: if you grew up sighted and in a culture in which eye contact was meaningful, you will likely think eye contact is a meaningful part of communication, even if the person you are speaking with has a completely different take.

that's just how things work. the same way folks without smell don't have any particular need for it, while for others smell creates intense emotional connection. just because something is not important to you doesn't mean it is not important to others, and vice versa.

is the article taking a fairly narrow view of communication? sure. it's video conferencing - the whole point is that people are looking at each other... but not quite. it's an uncanny valley! (a metaphor which probably works in multiple if not all senses)

i wonder how/if parallax eye contact impacts people using sign language?!

I'm genuinely curious: Do you use video chat a lot? If you do, is that a personal preference or something of a standard workflow at your job / you agreeing to someone else's preferences? My sample size of blind people I know and feel close enough with to discuss limitations/day to day issues is tiny unfortunately, so the question of whether video chats are a thing never came up (We did watch(?) movies before though).

The reason I'm asking is that I do agree with the article, but I'd phrase it slightly different: The technology right now makes video chats ~different~ and awkward due to the lack of eye contact, IFF you're used to eye contact in face to face conversations in the first place, because video chats are supposed to allow f2f over remote distances.

Thanks for asking. No, I don't use video chat much at all. Really, I only do it occasionally with my family when I can't be there for somebody's birthday (I'm in Washington state but the rest of my family are in Kansas). And of course, I only use video for their benefit, not mine.

I currently work in an on-site office environment, and only a small number of us are blind or low-vision. When I'm initiating a real-time conversation with someone, I'll often do it by IM. We may do it face-to-face if the other person is right there, but to me that's just like an audio call with 3D positioning, perfect fidelity, and no latency. To be fair, that is indeed still better than current VoIP, but not enough so that I would insist on being in the same building as my coworkers. Indeed, with an online audio group chat, I always know when someone joins or leaves the group; that's not always the case in person.

At my previous job, by contrast, we were an all-distributed team made up almost entirely of blind or low-vision people. As you would expect, we never used video. But we did use audio a lot, both one-on-one and group chats. (Our favorite voice chat system ended up being TeamTalk [1].) I never met most of my coworkers in person, and I never felt that this was a problem. It only just now occurred to me that the few sighted people on our team might have been uncomfortable with the way the rest of us worked. As for the rest of us, as far as I know, none of us ever wanted to be together in an office building. At least not for the sake of communication; having a shared pool of test hardware would have been nice from time to time.

[1]: http://bearware.dk/

While an interesting perspective, I think it does not reduce the importance of the eye contact for capable people.

When people capable for eye contact talk to each other and are not having eye contact then it creates an impression of not being there, or not valuing the conversation enough, or not wanting (or not being capable) to tell the truth.

Current video conferencing system creates an environment where participants do not necessarily feel comfortably. It is not equal to the real life conversation.

You can mitigate this by looking to the camera when you are talking and then to the screen when you are listening. This creates an impression that you are looking into the eyes of the listeners but then you can not truthfully see their reactions.

One option is to capture the video with depth as well as colour data and then just rotate it to point in the right direction(s) to maintain eye contact. This is what we experimented with on https://matrix.org/blog/2018/02/05/3-d-video-calling-with-ma...

I really don’t think looking someone in the eye is as important as it sounds. Yes facial expressions communicate a vast amount of information but I can still see someone’s face even if they are not looking me in the eye. Tbh I actually don’t like looking people in the eye for the whole of a conversation; it would feel weird. For what it’s worth, I don’t feel like someone is being dishonest if they don’t look me in the eye. That kind of myth can go in the bin along with the idea that a firm handshake shows confidence. That kind of thing makes me cringe.

One of the simplest things would be to simply allow the user to put their thumbnail at top center, where the camera usually is. Even better, maybe position it to where dynamically to above the head of the speaker on the other end. In my case, for example, the geometry of my desk leaves the front edge of my microscope visible in teleconferences, so I am no doubt slightly left of center in all cases. In any case, I know that I for one spend a non-trivial amount of time looking at my own camera feed, seeing if the video is ok: am I dropping frames because of a visible ceiling fan? Is the sun coming through a window behind me, blowing me out?

... I think most of us would say that’s a big improvement over audio-only communication.

Never assume that just because you have a particular personal preference that most people share it.

Not enough information to say that's the reason for the assumption, though.

I consider a well written email to be a big improvement over audio only communication.

Eye contact would be brilliant but at the moment I would settle for full-duplex audio. It really impairs the ability to have a natural conversation in e.g. Google Meet when you have to wait for the other party to be totally silent before you start speaking.

Link to one of the more recent and interesting projects mentioned: http://sites.skoltech.ru/compvision/projects/gaze-correction.

This would be a crazy difficult (and fun) domain to work in. Humans are very well tuned to perceiving facial details - the margin for error would have to be tiny. I'm still uncertain if images processed this way could escape the uncanny valley, but this appears to be a much more technically feasible approach for consumer use than transparent displays, multi-cam setups, or camera arrays embedded in displays.

If I were a betting man, this is where I'd put my money. It seems attainable via software or hardware assisted processing once the model has been computed. All the other solutions seem too expensive or hardware intensive to me.

I'm a full time remote consultant. I've noticed anecdotally that over the last year 80% of my contacts have started muting their camera all the time. Some cite "bandwidth" or other silly excuses; most just say nothing.

I'm as insecure about my appearance as anyone else- what's the deal? I keep my camera on until it's obvious that nobody else is going to show their face. Then I have to be the awkward one to turn off.

> 80% of my contacts have started muting their camera all the time...what’s the deal?

I do this. My camera is always broken.

I’m fine with audio, don’t like cheapening face-to-face meetings with video, and don’t see the reason to get ready for a remote meeting to which a video of my face adds zero value. It’s a preference which costs nothing, so I exercise it.

(If it’s a negotiation, why would I give my facial expressions away for free?)

I do a mix of video chat and conference calls, I don't feel the face cam really adds anything, but it is nice to have the ability to share the screen to review information, which is a reason to do video chat with the cam disabled.

why is it awkward?

I don't then have to get out of the bath.

I've been doing video meetings for years, and I've seen video muting go up and down based on team expectations / social morays, general engagement, and the kind of meetings (e.g., one where lots of people are just trying to get some situational awareness, but aren't really participating).

I personally get annoyed when people video mute by default, and when I was a manager it was something I would bring up in 1:1s. Lurking on meetings is fine, but if you have a real role in the meeting and you aren't multitasking a meal or something, I feel strongly you should have your video on and be engaged, and most of the time when people are video muting it's because they are hiding their distraction and lack of attention, or else it's reached a critical mass where people are muting just to fit in.

Now that I'm an IC again generally the best I can do to encourage people to unmute is just name them: "hey Joe, are you there?" even though I know full well they are connected. It's admittedly a bit passive aggressive, but it's also a way of gently telling the person they aren't being present, and it's unlikely I'm the only person who notices.

I also mute myself for the wrong reasons, so I'm not trying to make this an indictment of individuals. But it's so easy for people to drift away from their work, especially when you are remote, and we need to look out for each other and not just accept dysfunctional behavior.

Wow I hope we never work together. A mature team trusts one another and assumes the best. If I got that feedback from a manager in a one-on-one I would put in notice that day. What a horrible attitude to bring to your work interactions.

If you had a private conversation with your manager, and they asked you to "be more present" at telemeetings via unmuting your video stream, your response would be to QUIT? Maybe before that point you should talk through the topic with your manager. Surely the best result for everyone is to have a shared understanding, rather than making what seems like a minor communication "difference of opinion" a dealbreaker.

This is not a simple communication breakdown, it's assuming bad faith without any evidence. That is a red flag of a toxic work environment. Don't let your manager gaslight you.

You seem to assuming as much bad faith as you believe the OP is assuming. Is this (edit: your preference re: muting) a generational thing? Because I definitely see it as a reasonable expectation that somebody at a telemeeting would be visible-by-default, and only muted in specific circumstances such as large meetings, or after citing some abnormal local situation (e.g. not "presentable"). I believe body language is important and generally find communication still has "dimensional" improvements in efficiency, moving from text chat to voice to video+voice.

However, I can also see that my expectations are not shared by you, based on your strong response. My gut tells me this is related to different communication norms with "digital natives." I think your assumptions of malice and inability to learn on the part of the manager are misplaced.

You seem to be assuming both my preference and age.

I'll allow that some of it may be generational; many of my younger colleagues have never called in to a conference call.

But I challenge your claim that I'm assuming bad faith. At no point did I question the motives of the OP, only their actions.

Edit: I should clarify, because my original comment could reasonably be read as assuming details about the internal motivations of the OP. Allow me to rephrase: if a mangers' default response to an individual not having a camera on is to dress down the person for not "being present", that manager is making a toxic workplace environment.

Yeah I suppose I'm assuming your age. I don't understand where you're coming from when you feel that the immediate response from the situation is to quit, and my "maximum likelihood estimate" is that it is millenial vs gen-x office culture, hence the leading question. Apologies if that has offended you.

Your preference seems... clear enough to me? given that you said you would quit that day. What am I misunderstanding?

I believe your response of leaving rather than attempting to reconcile the difference in expectations is an indication of assumption of bad faith. As in, you seem to believe the manager would not be responsive to a discussion.

I've never gained anything from watching "body language" during a video call. A key component to body language is what the other person is looking at. But – you can't tell in a video call! I can't tell who's looking at whom or what – everyone is just blankly staring at "me". And half the time, I'm looking at relevant documents in another tab, so (a) I can't see anyone else's face anyway, and (b) the fact that I'm looking away from the speaker isn't reflected in the video feed.

Growing up pre-internet blessed me with the ability to communicate effectively without body language, thanks to the telephone. I find that translates perfectly well to video-less conference calls.

Yeah, all this looking each other in the eye and such seem very much like a boomer/MBA thing.

This is butts-in-seats mentality, but for remote workers:

> most of the time when people are video muting it's because they are hiding their distraction and lack of attention

The OP assumes that those who have their camera off are wasting time, because they can't see them to prove that they're not.

I get that, but... butts-in-seats as an expectation for the work day (which is typically where the policy/work culture is called out), is quite different from a butts-in-seats expectation for meetings where one has a "real role".

Can you clarify your position here? What do you see as the main issue with butts-in-literal-seats, and what about this hypothetical remote meeting is "quite different"?

GP cited "butts in seats mentality" in a way that I interpret as pejorative, referring to office cultures where physical presence during working hours is more important than output.

A meeting is specifically about communication. A participant at the meeting that is inattentive is either wasting others' time, or should probably politely step out, or at least should not have been asked to attend. Being present is a priority for a good meeting, hence the pejorative "butts in seats" comment doesn't make sense to me.

The output of an interactive meeting is the interactivity, not the appearance of interactivity.

I don't disagree with this; I just think dropping the video feed can impact the interactivity of a meeting.

" I feel strongly you should have your video on and be engaged, and most of the time when people are video muting it's because they are hiding their distraction and lack of attention, "

You shouldn't be projecting your personal opinion on others. Even if people don't pay attention maybe you should consider that it's because the conference is not that important to them. Happens to me all the time. If I paid full attention in every meeting and call I am supposed to attend I would get nothing done.

This is a very distrustful, almost adversarial, position to take. If you don't trust that your employees are paying attention just because their cameras are off, then you must not think very highly of them. Regardless of whether that's true, that's how that stance comes across.

I would not enjoy working for someone who thought like that.

Others have called out the extremity of this position, so I'll leave that alone, but do consider the practical/technical limitations here. Whenever there's a call with more than one or two speakers with a going video feed, the quality of the stream degrades precipitously. Depending on the quality of the teleconferencing software, either the whole call will become entirely unworkable and audio will get mangled, or the video streams will degrade into mush anyway.

Because of this, unless I'm expecting to be the primary presenter in a meeting or unless it's a 1:1, I'll usually "mute" the video feed. If I don't start video-muted, I'll frequently do it mid-call as people complain about lag, choppy audio, or inability to make out the primary presenter's screen. There's no reason to put the extra technical tax on the call just so people can see what I look like when I'm staring at a screen.

Please consider this as an alternative motivation, rather than assuming that anyone who isn't making themselves visible is slacking off.

In my opinion people tend to mute their cameras more when the chat involve big groups and/or third party to their team.

If is a team call or a small group trying to solve a problem, the chances to have the camera on are higher.

In essence, the camera gives you a more intimate feeling, and depending on the situation people are more or less inclined to it.

> muting their camera

Shouldn't the verb be "blinding" in this case?

This is a personal anecdote, but I don't mind meeting people in person. I can do face to face conversations all day long. The minute you point a camera at me though, I shut down socially. I'm suddenly extremely self conscious about my personal appearance and my behaviors and mannerisms, and that stress makes the whole thing unpleasant.

I'm not sure why this is. I think it's not helped that subconsciously, video feels one way. A participant can be looking right at me but I can't know that, because I don't know where my face is relative to their camera. So, I assume that all participants are watching me all the time, even if that's an irrational assumption, and even though 99% of them won't care if I scratch my nose or something.

I video-mute for the same reason. My face adds nothing to the conversation, except to make me feel self-conscious. So I video-mute so I can give the conversation my focus, rather than my appearance. I don't want to take meetings in a panopticon.

Your face does add something though. We've evolved to be visual. If I'm just meeting you, an image goes a long way in cementing who you are in my brain. Seeing your expressions helps me gauge my message's reception. Is it absolutely required? Of course not. It is an additional value add to communication. Just like voice adds context you can't get with text.

Personally I don't want to see the other person during a video conference. I find it distracting.

Video takes focus. If I know I'm being watched I end up spending a lot of focus making sure I'm sitting still, keeping my face neutral and looking into the camera. With the camera off I don't have to care about those things and I can focus more on what is being said and what I'm saying, or tune it out and get on with my work if its a group discussion that turns to a topic I'm not part of.

Eye contact is something I constantly need to remind myself that people feel is really important, but I cannot say I’m naturally aware of its necessity.

Yea, I find it really tricky to both think about what I'm trying to say and focus on maintaining eye contact.

Combining Tobii eye-tracker with some software (machine learning) based correction system might be interesting project to try.

You could collect training data by setting up webcam through teleprompter, regular webcam and Tobii on a machine. Just record normal usage and then learn a model using eye-tracker data and regular webcam data as input and data captured through teleprompter as target.

With Tobii the system would also know who you are looking at on the screen. With custom conferencing software, you could deliver the software modified eye-contact stream only to that person.

https://gaming.tobii.com/product/tobii-eye-tracker-4c/

Does anyone wish we'd just attempt to have fewer video conferences instead of trying to tweak this terrible technology?

I am really amazed that each video call app doesn't have an option that adjusts look of the eyes of the person you are talking to so they seem to be directed at you.

There's no magic in how people look when they look at you. All the magic of feeling when someone is looking at you happens in the recipient's brain.

eyes are the w̶i̶n̶d̶o̶w̶s̶ linux to the soul

This will be solved when in-display cameras become ubiquitous. Probably the entire display will be the camera, without a lens obviously. Using some nifty algorithms you can construct a focused image from that. It'll just be a layer inside the display.

Reply on news.ycombinator.com