People are already doing that, and have frequently lost track of the fact that it does not mean that they have gained any skill or knowledge, nor do they have an actual way ot ascertaining whether when there is a correct answer the processes used, which they do not understand, is able to reliably reach it, especially edge cases.

Sometimes the outcomes are comical. I just received an automated email from ElevenLabs saying that its automated systems have detected that I may be using its services to create voice versions of materials that harm children. I had it prepare audio versions of several books and academic papers about moral panics that conjured up out of nothing... about harm being done to children. At least that's what I assume. Either that or somehow I had an API key leak from my on-premise homelab, but the usage recorded would mean that they are basing a semi-conclusion based on a tiny sample. I have no distributed any of the outputs, I own the books and have access legally to the studies, because I have a background in the humanities. I also have no children, which ElevenLabs better not know, although how studies of moral panics can cause any harm in children of any kind is literally unimaginable. It's a waterfall of potential errors summed up in a vague email. My API key and the web interface works just fine regardless.

It's one thing if this is a product in beta, but this is their production model. Harming children is a serious accusation except the legal concept impossibility and the admission that this was not an actual lawyer (like I am) but some effective form letter hedging the vaguest of accusations made by some model lacking the ability to discern substance and meta-substance makes it frankly hilarious, and wildly irresponsible. I realize that by academic credentials I'm out of my lane but by experience I am certainly not, and they should recognize when they should stay in their lane and not just run Jev and think it's fine and dandy when done unsupervised (I presume).

Also, AGI is by definition asymptomtic surely, since there would be no way to benchmark it in a manner that isn't asymptotic. We're nowhere close to that. But we're so far from that, it's comical that people who clearly have zero idea of either the technical or conceptual aspects of basically a piece of software that is very good at quickly bruteforcing the correct or acceptable next token to be anything more than that. Even with some serious training and many hours spent on vast.ai I've yet to have created a version of a frontier model that is actually "good" at hacking in my own homelab setting. Although the the time stock Fable 5 missed a favicon shell on a basic jar (turned out they nerfed the hell out of it, this is why I only pay for the massively discounted tokens from Chinese proxies if I'm using American models or sometimes the freebies if you figure out how to get onto linux.do or similar sites). If anyone reading this is a high school English teacher, please inform your class (assuming the homeric stuff is still being taught) that there's no upside of being Cassandra but the record itself, and that should be enough.