Uncanny valley, here we come. Studies have shown that people are more at ease around a robot that tries to look like a cute humanoid robot, rather than a near-human but not quite construct.

Replacing the video feed of a person's face with a computer generated, almost-their-face-but-not-quite simulacrum sounds like it'd heavily increase user nightmare rates.

So the proper solution is to sit in the middle of a multi-screen panopticon and use multiple cameras on the telepresence device. People looking at the front of you would "see" you face left to look at something to the left, for example. You'd still have to turn the machine to face people properly if you wanted to make "eye contact".

Headphones would probably be a big help though.