Comment by bsenftner | Hacker News Reader

Comment by bsenftner | Hacker News Reader

Video AI acceleration is tricky, where many of the currently in use acceleration loras and cache level accelerations have a subtle at first impact on the generated video, which renders these accelerations as poison for video work: the AI's become dumber to the degree they can't follow camera directions, and the character performances suffer, the lip sync becomes a lip flap, and the body motions are reduced in quality, and become repetitive.

Now, I've not tested TurboDiffusion yet, but I am very actively generating AI video, I probably did a half hour of finished video clips yesterday. There is no test for this issue yet, and for the majority it is yet to be realized as an issue.

Out of curiosity, what do you do with the footage? in a personal way I found it fun for the occasional funny situational video, or for some small background animations, but not so useful as a whole. I understand for things like making sketches from scripts and quick prototyping it's nice, but I am genuinely curious what's the use :)

Also interested, since that is my same impression.

I'm creating a college / corporate seminar level class with a 3D animated host and instructor. Each lesson begins with a brief animated intro, which the student then reads their lesson afterwards, and then there is a chatbot that understands that lesson that engages with the student afterwards. The course will have about an hour of 1-2 minute videos in the end, and because they are an animated "professor" it is then possible to create other ethic versions, and other language versions easier than otherwise. And for the curious, that hour of final in-use video will be sourced out of somewhere around 8 hours of final video; these video AIs are fine and dandy for short content, but when working on longer form media the consistency issues grow to Godzilla sized and really become the most consistent issue: trying to keep the likenesses of the characters, and the environment from drifting over time.

Reply on news.ycombinator.com