'Megalopolis' trailer's fake critic quotes were AI-generated
variety.com
variety.com
Despite my PhD being in NLP, they still seemed incredulous at my assertion. I truly think the general public have not been adequately warned that they cannot trust the outputs of LLMs. I wonder how long until that fact becomes commonplace.
Customers don't want to pay for an AI to make a SME's life easier, they want to pay for an AI so that they can stop paying the premium for for SMEs.
When it starts killing enough people to make the news.
That isn't really the same thing. I was thinking of something more along the lines of "AI designed car randomly explodes" or "AI doctors misdiagnose patients and mix up prescriptions." The AI equivalent of the Therac-25[0]. Something a lot worse than "sometimes the media lies," which we already know, and something explicitly the fault of AI confabulation and automating away humans at what should be necessary stopgap or validation steps for the sake of streamlining or cost-cutting.
>If anything AI creates deniability, "Oh our so and so used gpt, woops!"...
Luckily, that doesn't seem to be the case. In every story I've seen where people have tried to blame AI they didn't get away with it, because the AI is just a tool and the human beings are still legally responsible. People still try though.
[0]https://hackaday.com/2015/10/26/killed-by-a-machine-the-ther...
(This phenomenon is not unique to AI, it shows up in all areas of life.)
Frontier models are better than they were and "feel" fairly reliable, although all the AI problems of 2021-2022 conceptually do still exist.
It hallucinated an Apple API call. Not just that, but that nonexistent call was the entire fulcrum of its "solution."
I’m pretty sure that a number of folks extended the API with this property (which is what I did, once I figured out how to address the issue. It just made sense to do so), and ChatGPT read that as a standard system call.
That’s likely to be an issue with more mainstream languages, as extension is a fairly common pattern, these days.
I think I may start tagging my extensions (like a naming prefix), to signal they are added after the fact. I do a lot of open-source work, so it’s likely to be used as training data.
The more strict your requirements, the less reliable LLMs are.
And while everyone here are going to laugh at those chumps who use data driven stock price indicators or LLM code generators, they'll throw a fit if you tell them that the data on calories and vaccines aren't always that accurate either.
to the contrary, every company now advertises using "AI" (the latest handle for LLMs) to "solve X" with maybe some small print somewhere about the side effects, much like medication advertising
Did you see Google's ad during the Olympics? No mention of hallucinations that I recall (also probably the stupidest use case for LLMs they could have thought up; pretty shocking).
I watched a presentation by some psychology researchers last year where they were presenting their paper for applications of LLM technology in early-therapy diagnostics. They were literally throwing patients at it. I showed one of the enthusiastic followers, a high ranking PhD in the field, how easy it was to get it to give me suicide advice and she was absolutely horrified and totally unaware of the possibility this could happen.
All these things were missing in the associated paper of course. Gotta keep those grants flowing.
A rare human would easily admit their mistakes. Especially when their career and money and power is at stake.
I've the feeling the term "AI" erased in a lot people mind the fact text generation is a classical NLP text; they actually had no clue what NLP is in the first place. So we became to the current absurd situation where opinions of people with actual expertise and knowledge highly related to the thing are dismissed.
On the one hand, it's surprising that none of the people involved in creating, publishing, and marketing this trailer raised an eyebrow. I don't know much about film history but I raised an eyebrow. And presumably, it took MANY people to make this trailer happen, and a good chunk went to film school or otherwise know enough to know that this was fishy.
On the other hand, things like this likely get caught during production all the time and we just never hear about them.
[1] https://en.wikipedia.org/wiki/David_Manning_(fictitious_writ...
It's a lot easier to believe the official explanation. Which is that someone typed in a prompt like "what are some quotes from famous movie reviewers that were negative about francis ford coppola's films?" and just copy-pasted the output without understanding that LLMs can make things up. And everyone who should've caught it was lazy or just simply didn't second guess the work that was done fifteen steps prior in the pipeline.
Yes, maybe some dingdong did this because they were stupid, and nobody bothered to check their work. But then you have to ask yourself: Why was a dingdong assigned to this task? Why was nobody competent assigned to check their work? And then hopefully you understand what's happening.
There's no "show me the proof that these are legit". It's assumed
I wouldn't expect anyone to check. One person in the chain made them up. Everyone after that took them as legit.
> On the other hand, things like this likely get caught during production all the time and we just never hear about them.
I seriously doubt it's anyone's job to check this. Most of society runs on trust. Things would grind to a halt if every person needed a chain of trust all they way down.
Unfortunately no, Google's results are now polluted with AI generated content, and Google search itself generates results with AI.
It hasn't been possible to trust the results of any search engine since around 2020.
Google is polluted with AI generated content but not this specific bit of AI generated content.
The error was "not doing it".
Faking your reviews with AI is not a good idea. The makers of Megalopolis got caught, and you could too. Their efforts to spin and deny won't reverse the damage.
Don't use AI for this. Don't even do this the old fashioned way!
Let's see if anyone is talking about this in two weeks. Attention spans are so short, all you have to do is ride it out for a little bit.
This is somewhat conspiratorial, but I think it's a deliberate enshittification strategy. Do something that's currently considered beyond the pale, accept some temporary reputation damage, walk it back. Then quietly and gradually start working your way up to doing increasingly bad things, and people will be OK with that, because at least it's not the really bad thing you did that one time. Eventually, the thing that was once beyond the pale is now the normal.
Coppola doesn't need a gimmick. He's famous and the film will sell some tickets just on his name. If the film sucks then it sucks. Don't make him look desperate in the process.
er...yes it was? he typed "WHAT ARE SOME QUOTES FROM FAMOUS FILM REVIEWERS ABOUT FRANCIS FORD COPPOLA’S MOVIES?" then copied the random text it generated and put it in to an email and sent an invoice for a jillion dollars to Lionsgate, who then put them in a trailer.
that's obviously an intention to fabricate quotes, and any social acceptance of it is just ridiculous.
Integrity is refreshing.