Animate Anyone: Image-to-video synthesis for character animation
humanaigc.github.io
humanaigc.github.io
Of course, the actual movement skeleton is coming from presumably real motion-capture, but still.
I'm curious what the current state is of generating the movement skeletons, which obviously has a ton of relevance to video games. Where's the progress in models where you can type "a burly man stands with erect posture, then crouches down and sneaks five steps forward, then freezes in fear" and output an appropriate movement skeleton?
But I think the more realistic approach is to take a video of yourself doing the motions you want the AI to generate, and run that through OpenPose instead.
Using just words leaves a lot to the model’s interpretation. I feel like you might wind up spending a lot of time manually fixing little things, similar to how you might infill the “wrong” parts of an AI generated image. It might be easier to just take a 15 second video to get the exact skeleton animation you want.
[0] https://github.com/CMU-Perceptual-Computing-Lab/openpose
Many complex moves (especially dancing) are filmed in video first, then the movement is traced over by hand. This is called Rotoscoping.
This is basically auto-rotoscoping, and I expect it to see commercial usage within popular high budget projects within 2 years. Previously, even the highest budget anime couldn't really afford 2d dance scenes due to the insane drawings required.
If this technology can instead be used to not only find the key poses but also roughly determine the inbetweens for a heavily stylized character like Rocko Wallaby or Blossom, Bubbles, and Buttercup then it is extremely useful.
If the "animator" is a hack.
I don't see any other way to smoothly link 1,000 possible movements to and from each other, including when there are various fixed distances between you and a ladder or ledge etc.
I think models will learn "movement personalities" the same way they learn a particular celebrity's voice -- everybody moves with different rhythms. So your big burly action-hero character will move with a totally different rhythm from your waif-thin ethereal elf.
But there will still be a textual vocabluary that generates the motion -- "stealthily creeps to the door 2.2 meters away, taking 6.3 seconds, and then suddenly and dramatically opens it with a flourish".
Look at tools like Adobe Animate and Character Animator, these tools are absolutely capable of doing interpolation between frames while giving the person operating the tools a fine degree of control. When you can use text to movement to quickly create tens or hundreds of sample scenes quickly, then manually iterate on and edit them, instead of hand drawing and compositing, the value proposition is pretty crazy for time and budget constrained production where tradeoffs on quality and fidelity are acceptable.
I recall seeing from-scratch learned realistic motion somewhere. I think was meant for character animation that would more realistically interact with the terrain and cross obstacles in video games. So that should be synthesizable too.
I imagine this will make the barrier to entry for animated stuff very, very low. You literally only need a character sheet.
Also, the creep factor for AI girlfriends has ratcheted up another notch.
I don't know if I agree with the framing here ("creep"), but for sure this will make AI girlfriend/boyfriend products even more addicting.
People who thinking porn is anywhere near the worst application are, IMO, letting their sensibilities cloud their judgement.
I want to show this demo to people, but I'm honestly slightly embarrassed to do so because it's such an obvious example of the male gaze. It's kind of tacky, and it distracts from the extremely impressive technology that's being demonstrated.
When I see all this generative AI stuff, most people are excited about generating manga, computer games and porn.
I know it’s exciting if you’re into consuming media. But I think the average person would look at this “breathtaking moment” and be surprised computers couldn’t do this already, honestly.
There is, when we're talking about characters made to look like children.
I don't know for sure, but that's my guess. You could achieve something similar with S3, but you need a credit card attached, and then you need to worry about whose credit card, and what if it gets unexpected traffic and who will pay...
You could use Google Sites as well, but then you need to buy a domain, which again means requiring a credit card, and whose responsibility is it to pay and for how many years?
I don't think it's mostly about the cost, I think it's mostly about just not having to link a credit card?
In my experience, it’s difficult to operate a small, personal website from within China because of their regulations in regards to non-government websites. Because of this, you will often find that Chinese citizens will use approved (or at least unrestricted) services like GitHub pages.
Having worked closely with many businesses based in China due to my hobbies, I have noted that services like Google Docs and Drive are favored for this reason.
I would guess there are ways to host content like this more easily on platforms that are only accessible within China or are not navigable without the user understanding Chinese language.
I would also guess that this is part of the reason why services that target customers in China tend to become “super apps” that combine several services that non-Chinese users would expect to find on disparate sites. For example, services may combine a social media style newsfeed/interaction API, banking, email, shopping, and food delivery into a single platform.
(I know people are often too exhausted to even upload the preprint... They take a break and upload later)
I don't feel that's an issue. A lot more people are able to see what's happening on the bleeding edge than if they'd just release the paper without accompanying demo page, and faster than if they'd wait for the code to be ready for release? Of course one can argue that "they should just release whatever code they have instantly", but that's their choice if they want to clean it up, remove secrets etc.
Bonus points if I can let the tool ingest season 1 or an OVA of said material where a season 2 is never going to come (looking at you "No Game, No Life")
Imagine someone doing this to show us "their" ending of berserk, for example. Just utterly disrespectful to Miura's legacy. The man dedicated his life to his art and you reinterpreted it like an ass hole because you lack patience and demand more content at the expense of all artistic agency. Complete trash.
Just a number really. That's all intellectual work is: a number. All numbers already exist, humans just discover them.
I just can't take it seriously, this notion that intellectual work is somehow "special" and deserving of "protections". To me it's as delusional as trying to own numbers.
> input of the author
Why do you need somebody's permission to do math with numbers? Makes no sense whatsoever.
> Imagine someone doing this to show us "their" ending of berserk
No need to imagine it when fanfiction.net exists. AI will make it even easier.
It's a fact that everything can be represented in binary. You can identify every knowable concept as either "cat" or "not cat". That's a fact.
Now... who cares? What does that being a fact mean? It means nothing. It's not useful. It doesn't prove a point any more than what you're saying does. All information is a number, that's a fact. Okay. Now... so WHAT?
The charge here is that some public information deserves protection. You disagree with that but, again, who cares? Prove why your opinion on that is relevant to any other human being, or engage with your irrelevance all by yourself. On the "protect public info" side, we can say that things like your social security number needs to be necessarily public, but we need to be able to stop people from using it however they want. There are indexing reasons for that. There are medical reasons for that. There are (yes, far too many) financial reasons for that.
And if you don't like that example, let's try one more directly applicable with a creative example: trade marks are public info. But if we let every person do whatever they wanted with a trade mark, they lose the real, tangible value that they provide (iconography is foundational to how human brains efficiently interpret information).
So pick you poison, but then engage with the actual poison; don't just wink behind smug tautologies. Tell us how, in your view, we can have a framework that provides the values that we get from IP, when we don't protect IP, as a concept. Because, buddy, I've been longing for, and working towards, that kind of framework since I figured out how pathetically inept IP is at actually doing what it does.
I care. So should you, and so should everyone here on Hacker News.
> What does that being a fact mean? It means nothing.
It means everything. It means what they really want is to more or less control what arguments you can pass to memcpy. The only way they can possibly do that is to destroy computing as you and I know it.
Think about it, about what they're really doing. All information is bits and copying bits is a fundamental computer operation. In order to restrict that, computers need to be made so that they will run all the programs, except the ones that invalidate intellectual property. Most likely they'll be made so they can run only the programs they approve. Government signed binaries, programming licenses.
Surely everyone here agrees that such a future is dystopian.
Which you might note is not an "answer" so much as it is succumbing to the chaos that you don't feel is important enough to control.
I do appreciate you making that explicit though! It's often hard to get people to admit to such naked dismissal of functional societal mechanisms that actual people get practical, tangible value from. Definitely contextualizes the framing more vividly.
Nothing wrong with that. Intellectual property proponents will "burn down" the entire technology field to get their way. Every day it's some new DRM bullshit. They couldn't care less about hacking and values like computing freedom. Then you have governments who want to do things like stop citizens from using strong cryptography.
The fact people currently get value from it doesn't justify its continued existence.
> succumbing to the chaos that you don't feel is important enough to control.
Controlling it requires destroying everything I care about. I'd sacrifice the entire copyright industry to keep it.
> I do appreciate you making that explicit though!
It's what I strived to do from the beginning. I want to see things clearly, as they are. If the conclusion is the current status quo is a lie and should be abolished, so be it.
If you find instances where I'm contradicting my beliefs, I give you free license to call me out on it in public. I will either try to justify myself or try to change immediately if I can't.
What I hope for you is that when you are reliant on something, it is stripped from you without regard for how you need it, and hopefully using a particular kind of logic that you find repugnant. That would be an appropriate reversal of the ideology you are preaching. I hope that you lose something you NEED and that the people that could help you simply won't. Not out of inability but out of ideology. I hope that it happens to you not because it would be distressing to you, but because I am confident that such experiences can make one understand how ideological purity (in any specific aspect, logic included) is not a particularly useful measure of the validity or utility of an ideology as it pertains to applicable function. And if it doesn't, then your argument is only enriched by having experienced it. It's a win-win, if you're interested in the type of rigidly structured nonsense you're expressing.
I'll take your silence as a realization of said pinnacle, and any future responses as indicative of your need for further education. And, in those cases, please understand that I am continually hoping for your success in that. Bonne soirée!
Man, this kind of tit for tat arguing is really not what HN is made for and it is very uninteresting to read, especially with thinly veiled insults like this one.
I’d love to know if you’ve actually ever created something novel that was good. Not code. Not math. Something uniquely original with no foundational basis.
If you want to see the stuff I've created, the links to my website and GitHub is in my profile. It's all free and open source software. I'm not a fan of advertising either so I try very hard not to post my stuff here unless it's directly relevant to the discussion. I don't know about "good", that's for others to judge.
The idea being that if we prevent people from doing specific math on specific numbers, the more proficient number-discoverers will have greater incentive to discover more of those pleasing numbers, and we all benefit.
... Which to me sounds wishful, given the reality of the system as implemented today.
Perhaps someone can help me understand the appeals to creative sanctity. Does making my own fan fiction cheapen the original work? Why is it immoral to change a fiction to suit my own preferences, as long as original authorship is not implied? I mod my single player video games all the time to enjoy them more. I have no qualms about patching out a tech tree if the grinding isn't too my taste, is this an affront? How about covering my favorite song? Making a custom cover for a special book in my collection? Skipping the scariest part of a horror movie? Singing a new jazz song imitating Armstrong's style? Help me understand.
That's a pretty big if. Think about what must happen for such violations to be prevented. You need computers that only execute legal software. It's literally the end of computing as we know it, the end of everything the word "hacker" ever stood for.
Creators are good but I certainly don't support protecting them at all costs. Certainly not at the cost of computing freedom. I love computers and I hate to see them limited to enable obsolete business models.
Imagine debugging your own code (your code being your intellectual work) and this guy barges into your room and says "Why are you wasting your time? Why didn't you just pick a better 'number'?"
There's a reason you can't copyright songs constructed through combinatorial algorithms. Human authorship plays a role in creative pursuit, by law.
No one said that.
> so why commit yourself to any form of craft
Because you enjoy it.
Intellectual property is all about controlling public information. They deliberately publish it out there and then expect to control its flow and what it's used for. Makes no sense.
Why do humans have any right at all? I can't take this seriously. Molecules don't have rights.
It's part of a very sensible legal framework to incentivize the creation and publication of hard-to-find numbers that you value very much.
I hate intellectual property as much as the next guy, but it's not nonsense.
> Just a number really. That's all intellectual work is: a number.
You saw that everything could be encoded in a number, and instead of expanding your view of what a number could be, it diminished your view of everything else. That's depressing.
I disagree about "sensible".
It's the 21st century, the age of ubiquitous globally networked computers. The copying, moving and transmission of numbers is literally a fundamental operation of these machines.
The tyranny required to maintain this "sensible" numeric ownership system grows every year. The world changing potential of computers is squandered because of this "sensible" system. Look at how much technology has been held back by the copyright industry. Are LLMs going to be yet another victim? I'm sick of it. Society needs to find another way to incentivize creation.
> That's depressing.
Absolutely. This state of affairs depresses me very much. To maintain this "sensible" system, the computer freedom we enjoy today must be destroyed. It's the antithesis of everything the word "hacker" means.
Intellectual property is an idea from last century or older. It doesn't have a place in this new century.
Those "sensible" systems will have to change. Instead of incentivising people to find high value numbers by this artificial scarcity, number searchers will have to find other ways to get paid. That's all.
People that pick up the trash in public parks don't get paid over and over for the beautiful work they did. They have to do it again and again as most of us, once the product of our time is out.
I think it's extremely disrespectful to make a TV show out of a work of fiction without the permission of the author. That author would be well within their rights to sue you.
Delusion is thinking you can own numbers.
What does it matter what these silly laws say? They're borderline unenforceable anyway. Imaginary ownership rules are constantly violated world wide at massive scales. Ever saved a photo from a website? Ever shared a picture with someone else? You violated these rules.
And what are they going to do about it? Are they gonna sue everyone? Throw everybody in jail for "disrespecting" their imaginary numeric ownership. What a bunch of bullshit.
It's the 21st century. Better to just accept it and let go.
You mean highest form of flattery? Literally participating and reflecting, trying to recreate and reinterpret is how humans integrate, show acceptance, make it part of themselves and culture. His art is not sacred, nor is it absolutely original, nor is it made in isolation from the world. Miura's legacy is sharing his reinterpretation of whatever. For it to exist there must be less elitist underground bunkers full of sacred IP which we unable to think about, discuss and share our interpretations of.
But, of course, when techies do the same thing that creatives have done for decades (IE, make transformative fan art), well now thats not fair apparently.
> you reinterpreted it
Reinterpretations, fan art, and "what if" scenarios are completely common in creative industries. People build whole careers on transforming others work.
At long last, _Firefly_ Season 2 is within our grasp!
From step by step repairing my electronics to learning about science it will all be customised according to my learning level and focus on things I want to know more. And hopefully without that a word from our sponsors section.
it won't have this, because the product placement would be seamlessly integrated into the content so naturally that you don't notice being subliminally persuaded to think or buy what they want you to!
Actually that tech would be kinda cool. The reality would be bleak, but the tech would be cool.
If nothing else have the characters look out of the TV and tap their foot impatiently like Sonic used to in the videogame when you stopped playing for a moment
The VR pr0n, the video games with dynamic AI characters. Dead actors and historical figures resurrected into movies and education.
I'm not so scared about my future nursing home now.
Even if we need 1000x precision to make it indistinguishable from humans, that's 20 years away. 16,000x is 30 years away.
By what measure of “ability”? And for how long can we expect that, and based on what is that expectation justified?
I think they mean Stable Video Diffusion.
I quote similar criticism from Dianne P. O'Leary:
> Suggestive pictures used in lectures on image processing ... convey the message that the lecturer caters to the males only. For example, it is amazing that the "Lena" pin-up image is still used as an example in courses and published as a test image in journals today.
That folks like you think advocating for diversity is about "political correctness" rather than building a world that is better to live in is very sad.
Diversity is fine. Forced diversity is not. Having different opinions and things that are not forcefully diverse is it's own form of diversity. Making everyone conform to your ideas is not diversity.
I mean, give me a break. They're completely tame other than being mostly women, but apparently that alone is enough to send people into a fit of hysterics.
This paper is what I'd characterize as "horny on main" (https://knowyourmeme.com/memes/horny-on-main). It's embarrassing. They've spent all this time doing impressive, cutting edge research just to make women (and girls!) dance for them. They could have chosen a dataset that at least made them look cool, rather than like weirdos. It weirded me out! Very likely weirded out others.
Citation needed
Though of course, that would've mostly been super uncomfortable if the field was also dominated by women.
None of the examples have people in their underwear.
Instead, it is regular people, and animated characters, doing tiktok dances.
Not sure why you are equating this to sexual content. Its simply trendy/viral content. Nothing sexual about it.
Dancing, and animated characters, are a great way of showing off what the technology is capable of.
Still, though, it could be useful in its current form, and making a more general system might mostly be a matter of collecting appropriate training data. Impressive work, just needs a more realistic presentation.
It's funny to go back to the first avocado chair or deep dream images that wowed me just a couple years ago.
I can't help but feel lost in the pace of tech.
Seriously all the people complaining about it being used for attractive women dancing... no shit. Have you seen life?
[0] https://github.com/CMU-Perceptual-Computing-Lab/openpose
Anyway, they will have to adapt. The diffusion models are not replacing e.g. live theater.
I'm sure there are women out there who don't have a problem with this. And I am not some kind of weirdo who has a problem with people enjoying looking at attractive people. But I think in a professional context there really isn't any good reason to lead with stuff that might alienate people. It is, at the very least, tacky.