I just watched a youtube about cruise ships. A near-riot broke out during a movie quote quiz because the AI that made the questions confused Casablanca and Gone With The Wind. If it aint ready for quiz night, it aint ready to teach kids.
LLMs work on tokens. They literally cannot see individual letters. Yet, unlike humans, they can instantly create a tool to solve this task for them and never be stumped by it again.
I'm so tired of this brain-dead gotcha comment.
Do you think your legislators or most users understand tokens and predictions and what is going on? Do you think that those who bombed a school using it know this?
Until frontier models can account for the disparity between what they can do and what the person (or other agent) is asking, we will continue to suffer bad information used like good information.
That tokens cannot count letters like that is not a brain-dead gotcha. Ignoring how humans are using these models is lacking in critical understanding of the real world. People are literally going to die due to humans not understanding how this works (and already have). True brain-dead, and body-dead.
People formed their opinions on LLMs in late 2022/early 2023 when ChatGPT was first released and taking the world by storm and then never updated them after nearly 4 years of improvements.
Hell, just being able to search the Internet did MASSIVE things for reducing hallucinations.