Hacker News Reader

Comment by mc32 | Hacker News Reader

Nice, I like the wear and aging element it affords the renderings.

This makes me wonder if at some point realism will extend to movement whereupon a studio begins to shoot (render) feature films involving digital "people" as costs undercut actors. Will they be able to cultivate favorite "actors"? So you see an "actor" in different and disparate roles? In other words create interest and following for a constructed "actor".

View on HN

Reminds of "The Congress" by Ari Folman. "Robin Wright is an aging actress with a reputation for being fickle and unreliable, so much so that nobody is willing to offer her roles. [...] Robin agrees to sell the film rights to her digital image to Miramount Studios in exchange for a hefty sum of money and the promise to never act again. After her body is digitally scanned, the studio will be able to make films starring her, using only computer-generated characters."

https://en.wikipedia.org/wiki/The_Congress_(2013_film)

https://en.wikipedia.org/wiki/Simone_(2002_film)

"When Nicola Anders, the star of out-of-favor director Viktor Taransky's new film, refuses to finish it, Taransky is forced to find a replacement. Contractual requirements totally prevent using her image in the film, so he must re-shoot. Instead, Viktor experiments with a new computer program he inherits from late acquaintance Hank Aleno which allows creation of a computer-generated woman which he can easily animate to play the film's central character. Viktor names his virtual actor "Simone", a name derived from the computer program's title, Simulation One."

Actually Happened (not a movie plot)

http://www.criticalcommons.org/Members/kellimarshall/clips/A...

"The late Fred Astaire 'dances with' a Dirt Devil vacuum cleaner. This controversial ad, okayed by Astaire's daughter but protested by his widow, inspired "the Astaire Bill," which was passed in 1999 to 'eliminate the exceptions and place the burden of proof on those using celebrity images, forcing them to show that their use is protected by the First Amendment.'"

I remember that commercial, and Audrey Hepburn's chocolate commercial. And Bill Clinton's role in Contact.

More recently there was also some news about actors having to double check their contracts to make sure they didn't sign away the rights to 3D scans of them being used outside of the film they were in.

> More recently there was also some news about actors having to double check their contracts to make sure they didn't sign away the rights to 3D scans of them being used outside of the film they were in.

Yikes! An actor works once and would never work again.

If that sort of thing becomes commonplace, that won't be an issue, because they'll never work at all, and actor will slowly but surely cease to exist as a profession at the high end. See the comments upthread about Hatsune Miku and such; Japan is much farther down this road than the US is.

Future contracts with the phrase "face and body royalties."

Creepy.

That reminds me of Novint (who made the Falcon haptic interface) inventing haptic copyright licenses and then buying exclusive rights from game companies for ~$40,000 each. http://www.hawkassociates.com/clients/press/full_news.cfm?cl...

Aki Ross from Final Fantasy: The Spirits Within (2001, https://en.wikipedia.org/wiki/Final_Fantasy:_The_Spirits_Wit...) was actually supposed to be Square's first CG actress for exactly this kind of thing.

Which turns out to be not anywhere close to what audience were expecting. I still remember When Toy Story was shown in 1995 created on Silicon Graphics, they said everything was so crazy good, that at the rate of Moore's law, in 10 years time realism in animation is finally achievable. 2005 and The Spirit Within came and we are still no where close. Fast forward another 10 years in 2015 we are definitely better, but realism and Digital Actor is still pretty much a fantasy. At least the case the audience knew it was not real actor. That is 20 years with Moore's law, now Moore's law is gone and we are slowing down, what other kind of innovation will lead us to realism?

The problem with "digital actor" is that, while we've got the "digital" part pretty well covered, that leaves the "actor" part which falls squarely under Moravec's paradox and probably (IMO) even requires consciousness to achieve.

They don't need consciousness, they need to quit ignoring the laws of physics when making things move.

They keep breaking the viewer's suspension of disbelief because they want to show their characters being so bad-ass that they defy the laws of physics. Instead of seeing people with super-human traits, they see really expensive cartoon characters.

If you look at movies along the decades everything is amplified. People have shifted their references. I get mental fatigue watching movies these days because everything is fast, quick, chopped, extreme. It's like a manga, without the childish part. High power doesn't mean high dynamic. For instance in Se7en, there's one single gun shot, the whole movie is tame but when that shot occurs the shift is so strong you feel it. It's a musical thing, what makes it is as much the notes than the silence between them. Balance.

I’m pretty sure there are multiple gunshots in Se7en. E.g. when they chase the guy through the apartment building.

Proof: http://cdn.collider.com/wp-content/uploads/se7en-john-doe-ha...

So I take this is at least the second shot ? damn fuzzy memory

Hah, I have the equal and opposite problem: I try to watch highly regarded old movies and find they're just incredibly slow and boring. (Certain kinds of movies work with a slow pace, but even supposed action movies seem to take forever to get to the point).

Some of them are too full. I must admit. But the other day I watched 3days of the condor and it was just right. All the presidents men too.

I still wonder if this is only youth neurons being drugged on all formats.

But realism was achievable and was achieved not that long later; Lord of the Rings is still a very good movie, you can start to see the CGI is a bit dated but for a long time, e.g. Gollum's character was the benchmark - worse, the Hobbit movies couldn't even top it, despite being made over ten years later.

Gollum, while very good, was never good enough to be said to be photorealistic or something. Your eye would not be fooled that easily.

Gollum was definitely a benchmark, maybe even still. Then there's Avatar and Cesar from the Apes movies. But we've yet to get close with actual humans. What started with Final Fantasy we still have that "uncanny valley" in things like Tron and Star Wars: The Force Awakens.

Side note: I can't find an article but I believe Gollum was one of the first times deaf movie-goers could actually read a digital actor's lips.

I find it weird how producers still insist on shoe-horning CGI humans into movies even when the technology clearly isn't there yet.

In Star Wars: The Last Jedi, Snoke is entirely CGI for no discernible purpose, and it's uncanny valley hell every second he's on screen. It's all in the animation, of course; the rendering itself is great, but there's something really off about his movements and facial expressions. What's odd is that they could easily have produced this character using a real actor in prosthetics, with some subtle CGI augmentations.

Tarkin in Star Wars: The Force Awakens is even worse, like something out of a video-game cutscene. I don't understand why they couldn't just have used an actor.

> Moore's law is gone

For CPUs, yes. For GPUs, not as far as I am aware. And rendering is the traditional GPU task.

no he is right. Moore's Law in its original meaning is gone, and with GPU we are just shifting goal posts to pretend it still applies.

Moore's law in its original 1965 meaning was an observation that transistor counts doubled every year and a projection that this would continue for the next decade. Moore revised the projection downward in 1975 to every two years, which remarkably has held into the 2010s. But it is transistor count, not necessarily performance. A coworker calculated that performance was doubling in 18 months. https://www.kth.se/social/upload/507d1d3af276540519000002/Mo...

Transistor counts continue an exponential growth curve, particularly in GPUs. It may be time for another downward revision on the rate (2.5 years? 3 years?), but we aren't getting as much single-threaded performance out of the increased transistor count. This provides increasing performance difference between the highly parallel GPU architecture and the more limited parallelism of traditional CPUs.

Wasn't the Moore law about transistor count PER surface unit? If that's the case, then even GPU are way below the predictions now, because chips have certainly more transistors, but their surface is also increasing.

Components per integrated circuit. The link above quotes Moore's 1965 statement.

I feel like motion and facial expressions will be a much bigger hurdle than raw graphics. The answer to when graphics is not good enough is to get closer and model the physical material more accurately and with more detail. Not necessarily easy but relatively straightforward. Then it's just optimizations like this article describes (bump mapping doesn't add something that 100x sampling couldn't get, but it does it 100x faster).

But body motion and facial expressions are entirely different. It's not like an optimization problem where you could just throw more computation at it and it could go away, it's just not possible to do perfectly at all. The best we've been able to do is track the motion of real humans and map that to a digital model; we can't generate it from scratch. We don't even have a general and convincing model of how motion should work. With graphics we've been through half a dozen models, and now here we are with PBR+extensions and the result is fantastic.

Believable body and facial expressions are going to be a much bigger hurdle than we've seen with graphics.

We do have a general and convincing model of how motion should work - we have video examples of "proper" motion, so we can optimize models with the explicit goal of creating motion that's indistinguishable from real motion with e.g. generative adversarial neural network models.

Things like https://grail.cs.washington.edu/projects/AudioToObama/ are just the start, we can generate facial expressions from scratch with some quality today, and we can work to make those generated expressions better and more realistic in future.

A sample is not a model. Taking a picture of a surface, however detailed, doesn't get you any closer to rendering it.

Neural networks are exciting here though. But even if it gets off the ground, it will still be at the "Phong shading" level model for some time.

A single sample is not sufficient for a model, but if you have enough samples to infer the whole distribution, then that is sufficient.

A thousand pictures of a particular surface may easily carry sufficient information to enable rendering a million different pictures that are not distinguishable from the real ones; a thousand motion capture sets of different people walking carry sufficient information to allow generating new motion sets that are different from the original samples, but not distinguishable from real people walking.

I used to think this, too, but recent games like Metal Gear Solid V: The Phantom Pain have changed my mind. Astounding graphics and facial expressions.

Curious what Kojima will do with Death Stranding:

https://gamerant.com/death-stranding-still-shooting-scenes-m...

https://www.rollingstone.com/glixel/features/kojima-death-st...

These are great, but it's still not generating facial expressions, just mapping real faces onto digital ones.

We've already seen various things in that direction: characters performing plays (e.g. Monsterpiece Theater), one show doing a pastiche of another (e.g. the Death Note episode of Absolutely Lovely Children), fictional bands continuing outside the show they came from (EARPHONES or egoist) or bands that are simultaneously fictional and real (e.g. the one in Love Live), free-floating virtual characters being pushed as "celebrities" outside a particular show (e.g. Kizuna Ai). Look at e.g. the Hatsune Miku cover of the Virtual Insanity music video.

Black Rock Shooter is arguably already an example of what you described: a character "played" by Hatsune Miku. (Similarly subsequent efforts like Mekakucity Actors).

You may already know about Hatsune Miku[0]. I imagine it's only a matter of time before we see this kind of thing in film too.

[0]: https://ec.crypton.co.jp/pages/prod/vocaloid/cv01_us

This can happen only if they solve the problem of facial expressions. The more realistic the characters look, the more we expect them to behave like humans. You probably noticed how weird animated human characters are, it's like something is off but you can't really tell what. I suppose it's the microexpressions that animators can't express.

When we start render digital "people" as actors, doesn't that mean that whoever is the pupeteer needs to be a good actor himself. This possibility poses some interesting questions. Can AI learn the script and, given some personality traits, behave like and actor? etc

I suspect that AI could help in this regard. Most precisely adversarial neural networks[1]. One network to improve realism, and the other to detect uncanniness.

1. https://en.wikipedia.org/wiki/Generative_adversarial_network

That being said, microexpressions have gone a long way already. But indeed,

> [...] whoever is the pupeteer needs to be a good actor himself

Example: https://youtu.be/jSJr3dXZfcg?t=212

Isn't that already the case in background actors that are not in focus in many cases? There are a lot of examples on youtube, where background actors are either digitally cloned or entirely created only digitally. This goes back at leat a decade, only a matter of time until it will move into the spotlight more. Planet of the Apes is a recent example of pretty impressive CG characters where only movements come from real actors, but of course these are not humans and still look CGish.

Could be: a lot of the 'branding' of the actor these days is about curating what films they show up in. Like, if Johnny Depp is in a movie, you know generally what kind of film it'll be. The AA (Artificial Actor, haha) would benefit from having an agent, much like real actors, to keep consumers' meta-dissonance down.

That said, actors swap between wildly disparate roles all the time and no one seems to care!

The advantage of digital actors is that you can break down the job into multiple roles.

voice acting; on-scene presence, body language and facial expressions (acting in the theater sense); stunts; physical appearance.

Check the movie S1m0ne :) http://www.imdb.com/title/tt0258153/

They can already do the next best thing, and has been doing so since at least the original Matrix.

Damn it, it has reached the point where ads can reuse long dead actors.

There was a HN post some time ago about a ML tool that can put the face of an actress on top of the face of a pr0n actress.

It is not limited to any sex, you can just as well map male faces on male actors.

https://www.reddit.com/r/deepfakes/

Reply on news.ycombinator.com