The emergence of full-body Gaussian Splat deepfake humans
blog.metaphysic.ai
blog.metaphysic.ai
But I would LOVE to see this transformed into a game engine/game dev IDE. Imagine if anyone with a smartphone could create photorealistic character models in under a minute. Hell, players could even provide themselves as character models ad-hoc without involving any devs.
It would be a renaissance moment for game development speed and accessibility, and make things like 'asset store games' irrelevant.
Imagine an LLM built to understand a base game engine template code, and also able to perform gaussian splat transformation on frames of input video.
It also seems like it could be the missing link for AR/VR crossing the uncanny valley.
If the recent history of tech is anything to go by, the most popular in-practice use case will be porn. All it will need is a corpus of training data: which it's safe to assume already exists.
After that, deepfaking $WorldLeader$ doing something embarrassing is probably #2 or #3. Just in time for the US elections...
I suspect in the upcoming election, deepfaking the candidates doing something out-of-character would have a similar surprising effect.
One frontrunner is about to be convicted and his corporate empire dissolved while the other frontrunner is facing an impeachment inquiry and his son is about to experience the full brunt of the Capitol Hill muckraking process.
Deepfakes aren't even going to register in this election (no pun intended).
This is the way it always was back when our primary means of communication was verbal and hand written. Everyone knew anyone could lie. We then had this bizarre period of time when you had certain types of media, especially photographs and video, that were extremely hard to fake convincingly. That period is now over and we have returned to the norm: you can't believe it unless the provenance of the information is known and the chain of custody is trustworthy.
This needs to be drilled into people starting in elementary school.
Which will result in different groups of people living in different realities - if what evidence you accept comes down to whom you trust as reliable, different societies (or sections thereof) will choose to trust different people
Which in itself is maybe a return to the historical norm - the truth of Catholicism was obvious to all in 12th century Rome, while the truth of Islam was equally obvious in 12th century Mecca
People have always been living in different realities, since there have always been people who believe false statements, and people who doubt true statements.
A big enough difference in degree, the same issue starts to look like a different one.
Convincing people that photos and video are as believable as a rumour is not as helpful as it should be.
It is very helpful to some people, namely those parties who flourish among the ignorant and poorly-educated. Remember that the real goal of disinformation isn't to make you believe false things; it's to make you believe nothing.
How are "people" actually supposed to know there's "a verifiable chain of custody to the information"?
I think this case is instructive to the workability of that idea: https://en.wikipedia.org/wiki/Killian_documents_controversy. A major news organization was fooled by a forged document that was later identified as such by amateurs. How is any organization supposed to have "a verifiable chain of custody to the information" for leaked information? That's the avenue for a lot of important information, and without it you'll just have a lot more reporting on press releases.
This seems like the hard part. How do I know that anything anyone is sharing is trustworthy? It's not like I'm going to trust any news org, government, non-profit, corporation, anonymous individual, etc.
What I think will happen is that it'll be like what we have now but worse: some podcaster will make an offhanded remark about Mark Zuckerberg "saying some horrible stuff about lefthanded people" causing frantic TikToks about that it means killing lefthanded people, all based on a supposed "secretly recorded video" that no one can validate due to all of the noise.
Recent history? I’m pretty sure as soon as the camera was invented, someone took a nude picture with it.
On the other hand, it has been possible for an extremely long time (millenia) already to create text of anyone doing anything, and in general it seems we can still trust text mostly (and can distinguish somewhat well between false and reputable textual sources), so maybe it's all reputation based, also for video in the future.
> But no federal law criminalizes the creation or sharing of non-consensual deepfake porn in the United States. A lawsuit is also unlikely to stand up in civil court.
https://www.wbur.org/endlessthread/2023/06/23/deepfake-porno...
> ... if someone makes a deepfake or otherwise posts a nonconsensual video of someone who does sex work, the defense could argue that because the woman’s reputation already involves sexual activity, her reputation is not being defamed by another video of sexual activity. Such an argument — and thus pursuit under defamation as a whole — would be missing the actual point, which is the violation of consent.
https://www.cyber.forum.yale.edu/blog/2021/7/20/deepfake-por...
This won't be regulated away, it will happen and everyone should be educated and prepared. That's just an unrealistic dream though, enough people will get fooled (I will probably be among them at least some of the time) that it will have consequences. Deepfakes have been used for scams for years already: https://www.youtube.com/watch?v=vqr0oER03SE
The situation is very similar to CP, and should be treated in a similar fashion.
However the adaptability of character creators is still a hard problem.
Like, I’m not particularly concerned with character aesthetics but I still found myself reject every single god damned hair style for my male OC in Baldurs Gate 3.
The most important thing is doing this in a way that works well with the lighting and of course being performant. Whenever you see something generating avatars, you need to question if it’s viable in game, because often it’s not.
No suit motion capture is already a thing. It’s not great, but it is a thing. Most games don’t actually need that many animations.
That said metahumans don't look much like the subject imho - and scanning with reality scan is a nightmare experience
I thought splats were explicitly not neural?
A Gaussian Splat, instead, is a neural representation unit that is not limited in this way – not only can it be assigned anywhere in XYZ/3D space, but it can as necessary multiply and subdivide into additional splats, as coverage requires.*
And the asterisk leads to the following explaination.
Technically it’s a rasterization unit rather than a neural unit, but in all current Splat implementations that are of any power or interest to the synthesis community, it ends up as a neural unit, passed through standard training processes.
Amended Friday, December 15, 2023 13:33:47 EET to clarify ‘neural unit’
But even with this I am not sure that I understand what they want to say.
Look at the domain name of TFA. They are saying "neural neural PageRank look at meeee"
I also think gaussian splats encode the same information as NeRFs, which have neural in the name.
For this post, they have more traditional neural networks embedded in the pipeline (search TFA for "decoder"), and they're thus calling the composite technique a "neural" technique.
As to the CNN/Transformer/... question, those are still very similar to a perceptron. A multi-layer perceptron has a sequence of alternating matmuls and activations. The CCN has a particular choice of sparse matmul and activation (and a couple choices of fast implementations to operate on that sparse matmul). A Transformer is one way to wire up many such MLP predictions (recurrent networks being another) to work with long/uneven inputs. There's more going on, but most of it is fluff to make the thing numerically stable and do something interesting. The core structure is still a dressed-up perceptron.
Would that not make fitting a degree 1,000,000 polynomial a neural method?
I also think gaussian splats encode the same information as NeRFs, which have neural in the name.
The collection of source images also encodes the same information but I don't think we should call them neural images.
I would argue we should stick with the original meaning, inspired by biological neural networks, a network of artificial neurons. Calling everything neural because of the hype will benefit no one in the long run as the term will just lose any meaning.
They also consist (in the abstract, at least) of interconnected neurons with activation functions.
Basically everyone was afraid to go outside and show their true nature and would curate from the safety of their home a perfect outward appearance.
I know religion doesn’t do it, that’s for sure. The only thing I can think of is some kind of authenticity.
I don't want to see this too negatively though. In principle the pool of women who stay fertile longer will outproduce women who lose their fertility too early. It also means there is genetic selection for longer lifespans in general.
All of what you said I do expect to be the expectation past 40 though.
I know many of you will disagree with me on this, but I also think a large problem is the increase in both extremely low frequency, and radio frequency, electromagnetic fields.
No one can tell me that the bloody nose I get whenever I use laptops that emit high levels of low frequency electromagnetic radiation is psychosomatic.
Hope the benefits will outweigh the dangers!