Hacker News Reader

Comment by ladyattis | Hacker News Reader

The Matrix demo is close but the facial expressions are still a bit off like the person who's doing mocap is wearing a mask even if they're not. It'll be quite interesting when they finally get those issues resolved and when developers also try to make more unreal (pun intended) looking effects that interact more with realistic looking settings or characters. Having the freedom to make such a disjointed thing in a 3d renderer like how surrealists use to do with oil paints and the like it something underutilized in cinema and games imo.

View on HN

Horizon Zero Dawn has the best facial expressions I’ve seen yet. First I noticed the lip synching was very good. Then I noticed it also seemed to be doing procedural micro expressions as well as small eye saccades. Paired with subtle and expressive voice acting it’s remarkable.

Are you being serious? Don't get me wrong, HZD is an amazing game, but man the faces are absolute robots. That's not the team's fault though - any open world game with lots of dialogue will pale in comparison to hand animated faces.

I’d be happy to view video from a game you think does it better. Also see my other reply.

Uncharted 4 comes to mind: https://youtu.be/KdDHJnqS32E

That said, I agree eye movement is a great way to convey feeling, and Horizon Zero Dawn did it well (and its DLC even better). Yakuza 0 is another example of doing a good job. The exagerated animation style also shows in the characters' eye movements. A great sample is this sequence that, very conveniently, shows a blind character that lacks eye expression. https://www.youtube.com/watch?v=jfVqfelfYHo

Last of Us 2, Death Stranding

L.A. Noire from 2011(!)

> any open world game with lots of dialogue will pale in comparison to hand animated faces.

Pretty sure this won't age well - or that it's already aged out. There's only so much animators can do. In the film industry algorithms often augment an animator's work. Often they do the bulk of the work, and animators will fine-tune to get the exact effect envisioned.

Also, artistic animations can look great, but they can also be very inconsistent. Even more so when you need multiple people to work on thousands of lines of dialogue. It also requires an enormous budget - not all studios will have Gollum-like funding for every single NPC.

Do you mean Horizon Forbidden West? Zero Dawn didn't have the greatest animations, but Forbidden West was a huge leap forward.

I mean HZD. Haven’t gotten to HFW yet. I think the animation below the neck is subpar, but not the faces. On a hunch I checked Last of Us 2. To the best of my observation, the eyelids, circular muscles, and nostrils don’t micro-twitch, and the eyes don’t regularly saccade. That’s what I’m talking about.

Forbidden West is going to blow you away. Even subtle sarcasm of characters comes through.

Need the go back and try to push on with forbidden West but man.. it was brutal even after just finishing ZD;not sure how approachable it will be after a long break.

Forbidden West is the sequel that came out in February. From "it was brutal even after just finishing ZD", it sounds you're mixing it up with The Frozen Wilds DLC that came out for Zero Dawn.

Yup, 100% got them mixed up.

Forbidden West has some of the best mocap with voice acting I've seen in a while since the God of War 2016 game. It's one of those things that's part art and part technology really, but making better tools will get this over the hurdle I think.

Do companies use machine learning for face animation at this point? E.g. capture a bunch of facial mocap data and phoneme streams and train a transformer model on how the face would move given a new stream? I guess it's probably easier just to translate phonemes to facial movements with much the same outcome?

For me, HZD (and most games really) facial animations really fall flat when the characters smile. They look so.. plastic.

Agree. Not sure how much could be blamed on the poorly written dialogs. In contrast, for example, Witcher 3 was so engagingly written that I never minded the procedurally animated dialog scenes.

For me, it was definitely a visual thing. As in, they look plastic in stills too. There's just this very unrealistic uncanny valley thing going on when characters smile that few games have been able to get around so far (but I haven't really played any latest gen stuff, so maybe its improving).

Young Aloy was worse than adult Aloy [1] but its like that for adult Aloy too, and most other games, although I found HZD worse than most in that respect. I haven't played Forbidden West yet, so I'm curious to see how much they've improved there.

[1] https://i.ytimg.com/vi/OeebacNhHIY/maxresdefault.jpg

The demo was great, but it really fell apart any time the mocap models moved. Despite the animation being great, the movement is still stilted and fake.

The next step change will have to be some form of advanced kinematic rigging that creates advanced motion granularity akin to the graphical fidelity of nanite graphical tech.

UE5 is a step in that direction, but there is still lots of work to be had.

These tools are still going in the direction of making a person sit in a chair all day.

What will really be a game changer is AI that can generate the bulk structure having been trained on our best of the best hand built models and we can iterate on the emotional details.

I’m working on the “bulk structure” part, training models to generate random game worlds that roughly adhere to the rules, look and feel like everyone favorites right now (while avoiding copyright issues).

After that my goal is empowering consumers directly to nudge the styles in their preferred direction.

I’m mostly motivated by the MBA-ifying of everything. My goal now is to just have AI produce new content for me, even if such a thing puts game developers out of a job. I’m starting to experiment with cartoons as well. Optimizing for myself like we do.

Absolutely true.

I still want to comment because your comment doesn't do that demo justice.

It's the first time I expierienced that awkward feeling looking into reality while watching the video of that city.

We are Eye opening close to cross over uncanny valley after reaching it that soon.

Recently Unity released a video with quite impressive facial animations. Unlike unreal, this version is not yet ready to actually try yourself yet unfortunately.

https://youtube.com/watch?v=eXYUNrgqWUU

Yeah that one is surprisingly good, even if the eyes move in a way that feels like the character can't see. I'm not sure why I feel that way, but something seems empty.

Facial detail may be great, but it’s amazing to me that they would put so much effort into making an animated character look life-like, but neglect to have it breathe.

There is zero chest wall movement. If you have a friend who works in medicine — especially emergency med or intensive care - send them this link and ask them how natural this looks. Bet most of them will immediately notice the same issue.

Huh - that's really interesting - thanks for that insight. One more thing to simulate.

The only thing that matters is that it's better than previous gen

Its possible that the Matrix demo was railroaded in the late parts of development by the purchase of Weta Digital by Unity.

Reply on news.ycombinator.com