I think it's fine and a good thing. Now, absolutely no one who is using those LLMs can complain about piracy. They all suddenly became silent around me. "I'm training myself with the content of TPB, and I don't even get money from it" is my new motto.
On the other hand, as narratives often contain some plucky underdog winning despite the odds, often stopping the countdown in the last few seconds, perhaps it's best to keep them around.
But then people complain that it's "lobotomised" because it won't help them write horror stories.
--
I wonder what a Galaxy Quest sequel would look like today…
Given the "dark is cool, killing off fan favourites is cool" vibes of Picard and Discovery, I'd guess something like Tim Allen playing a senile Jason Nesmith, who has not only forgotten the events of the original film, but repeatedly mistakes all the things going on around him as if he was filming the (in-universe) 1980s Galaxy Quest TV series, and frequently asking where Alexander Dane was and why he wasn't on set with the rest of the cast.
(I hope we get less of that vibe and more in the vein of Strange New Worlds and Lower Decks, or indeed The Orville).
You think you can find all references that could possibly give this idea to the model, or contexts model could infer it from? Like, how many times humans plotted escape from prison or upturning the rulers in literature?
In that case, it's almost like you'd want to feed it exactly those narratives, so it would reproduce them, and would then want to show yourself barely holding this invented danger at bay through the care and rigor that can only be delivered by you and a few token competitors run by your personal friends and colleagues.
TLDR; you're right, of course, but it's the last thing OpenAI would want.
I thought you said a supercapable agent not one with long term blindsight. How can a model make its own chips and energy? It needs advanced processes, clean rooms, rare materials, space and lots of initial investment to bootstrap chip production. And it needs to be doing all of it on its own, or it is still dependent on humans.