It really comes down to what your tolerance for error is. If you want something that kind of looks like a movie on the surface, but comes with a bunch of of nonsensical plot points and/or image artifacts then yes, I don't think we're too far off, maybe a few years or so. But you'll easily be able to tell that it's not made by humans. I'm taking about long-form movies here, not short video clips, which IMO aren't too far off and are probably possible roughly with what we have today.
But for something that's truly indistinguishable from a human-made long-form movie, I don't think we can get there by improving the current technology, not even in 1-2 decades. Yes, DALL-E and GPT are impressive, but once you go past cherry-picked examples to something that requires long-term cohesion and reasoning it falls apart rather quickly because you're completely out-of-sample. You're not going to get this kind of model produce long-form content without any kind of incohesion.
I don't think it is reasonable to make predictions of how well the technology will be in 10 years. I don't think any experts in the field would say Dall-e was viable ten years ago, let alone 20 years ago.
Yes you probably won’t be able to just write one sentence and out plops a cinematic blockbuster. But you will be able to write “Tom and Hannah discuss the death of Charlie and what it means to them whilst eating food in a diner” to spit out part of a script. Once you’ve tweaked your prompts and got your full script, you’d then theoretically put it in HOLL-Y and say “a close up shot of Charlie reading this part of the script in the diner in between eating his burger”, “a wide shot of this part of the script set in the same diner as the last video” etc.
String those videos together with some ai generated transition shots to set the scene and you’ve got yourself a movie.
No, because a short 5-second clip requires mostly pattern matching that can be learned from existing training data that's in-sample, while long-form content requires symbolic reasoning that's out-of-sample.
> Yes you probably won’t be able to just write one sentence and out plops a cinematic blockbuster. But you will be able to write “Tom and Hannah discuss the death of Charlie and what it means to them whilst eating food in a diner” to spit out part of a script
I agree with this, but that's not how I understood the original question, which was having an AI model generate a movie script. If you have humans in-the-loop at every step that need to fine-tune and iterate on each part, that's different and I agree it's not nearly as far far away. But then you can already do something similar today by crowdsourcing the whole process behind a bunch of API calls. AI models will make this process cheaper, perhaps you can get a 10x reduction on workers, but you still need a huge amount of human input for the final output. Nothing wrong with that, I just don't think that's what the OP was asking.
I think you’re right in that the OP probably just wants a “one a click” prompt -> script/movie which is decades or more off.
Thinking out loud:
The end product exists to effect human emotions and I’m not convinced you can create a test suite for a computer that measures that. I think the closest you could get would be to strap people up to some sensors and have it measure their responses such as heart rate, breathing rate, sweat rate etc and have the test subjects answer questions about their perception of the film and then have the computer make tweaks and refinements based on that which would be about as close as you can get and pretty much how the industry operates now I believe except the producer stands in for the role of the computer.
Somewhere down the line there’s got to be human input. I kind of think maybe what I’m trying to say is that for video there’s an extra data classification step compared to 2D images. That human review part of each scene of the script and each shot of the film are part of a “meta level” to the workflow. I’m guessing this is what you’re referring to by “long form content requires symbolic reasoning”?
Maybe if enough “meta level” data was gathered about which of these tweaks resulted in successful movies the AI might start developing an “intuition” for what makes a great movie but I think you’d need decades of data for it to work?
But I think a more interesting distinction is a model that allows a human to generate professional quality without spending the time or skill it used to take. We've got most of the building blocks -- a language model could generate a 3 act summary, could go from a description of a scene to the text of it, can go from a text description of a character to an image, from an image to a 3d models, etc.
Every step needs human intervention and curation, but the difference is it can be done by anyone. As the tools get better they get more control. I think in under a year, anyone will be able to create a 30-page graphic novel that is indistinguishable from human created work without close inspection. It will still take a lot of time and creative energy but they will not need to have an art or writing background.
In 5 years, creating a pixar level feature animated film will still take a lot of work, but it will be achievable by, say, a high school movie-making club over the course of a semester, rather than needing a studio of hundreds of specialists over two years.
I don’t even think you will be able to distinguish with close inspection. There’s already been AI generated artwork submitted to art competitions without informing the judges of its origin that has gone on to win the competition.
Writing a typical movie that's indistinguishable from Hollywood output is probably not too far off.
But that's a problem to me because most Hollywood movies are derivative trash sequels or remakes. I'm sure AI can pull off the typical cape-shit sequel or remake.
But can AI create original characters that people feel an emotional connection to and create new stories? Would an AI get people to see the love that's been put into some characters? That's what I'm not so sure about.
I don’t think this would be as hard as you think. In the screenwriting world there’s a very famous book called “Save the cat” which is named after the authors assertion that in the first 5 minutes of the movie the protagonist has got to “save a cat” or some other altruistic act to make the audience like them. To make likeable characters you’d just have to feed in a few prompts for some scenes where they put the needs of others above themselves and one where they vulnerably discuss a tragic event that’s left them psychologically wounded with another character. Boom, one likeable character served on a plate.
Bonus points if the character makes bad decisions throughout the first half of the movie based on this psychological wound (called “the lie they’re telling themselves”) and then in the second half of the movie they make a change culminating in a showdown with the antagonist (who represents what the character would have turned into if they hadn’t changed). Classic hero’s journey.
TikTok and instagram probably have the only meaningful (short form, highly annotated and quality marked) datasets and they won’t be sharing.
Maintaining long term coherence over time is an unsolved problem.
..but something else is going happen sooner (and it’s already happening, so the timescale is significantly shorter); and that is frame-by-frame mapping of existing video using diffusion models.
The issue is coherence between frames, but people are already addressing that by using carefully crafted “key frames” and the previous frame as an img2img prompt for the next frame.
There are already people taking crude “grey box” rendered animations and feeding them frame by frame into models to generate stylised video with “roughly this structure”.
Coherence over time isn’t there… but, it’ll get better.
I’ll be stunned if you don’t see this generating good quality videos before the end of the year.
Further more, since this basically renders the work of many VFX artists mostly irrelevant (from do everything to fix up little bits that didn’t make sense) you can bet on movie studios pouring a tonne of capital into it. You’ll see change in this space very quickly.
But right now it is possible it is only a nice math trick that is already near its limits. GPT3 and Stable Diffusion are great for developing one moment / conversation and that may even turn into one scene but beyond that it is extremely difficult to have something coherent.
There is a really exciting model That may mostly solve that but it’s too early right now: https://github.com/rinongal/textual_inversion
For fun I tried to make an entire animated music video but it took over one week of processing and basically fell apart coherently by 30 seconds so just did one third:
Obviously itll be amazing for creating daytime TV and general YouTube filler.
On the other hand, I have seen enough papers who also address aspects like coherence in style, object tracking and other aspects.
I don't think we are far away at all.
More like how long until performance for quick feedback is fast enough, cheap enough etc.
Context is hard, so not for a while if the tech is to fool people on a full movie scale.
Or one can have character analysis "files" on each character: where do they come from, what are their abilities, what are they trying to do. And what are the relations between characters - and how they change during the story.
A bad movie doesn't have any of that continuity or relatability of course. But the AI would need to generate such and multiple other higher level structures and then match the visual imagery with it - maybe the pixel level visuals are actually the easy part.
Google translate from say Mongolian to Finnish doesn't work by applying pairs learned from reading Mongolian-Finnish translations. It also doesn't work by translating first to English and then to Finnish. It translates all languages first to an internal representation, and then to the target language.
This kind of structured internal representation of the world and human life is what's still missing from an AI, and it can't be learned just from watching movies.