> And then you'd ask it about an area you're knowledgeable in and realise it routinely makes stupid mistakes.
This honestly doesn’t happen to me much anymore. In what areas do you find LLMs routinely make stupid mistakes?
This honestly doesn’t happen to me much anymore. In what areas do you find LLMs routinely make stupid mistakes?
It's objectively very difficult and technical, it's spatiovisual, it's artistic, learning resources for it are sparse and most just learn by the FAFO method, current AI sucks terribly at it, and it's not likely to ever be specifically targeted by benchmaxxers.
- Created useless pydantic schemas with all fields Optional[Any]
- Created a REST endpoint that silently mutated on GET (unsubscribed users from a mailing list)
- Failed to log costs in my app so users could have bankrupted me, etc, etc.
Good job I actually review its code.
Or, as someone else points out in another thread here, academic writing. It's one of the things newer models seem to have actually gotten worse at. Even when you give them detailed instructions on how to write and what to avoid, the "load-bearing", "A but not B" and journal-like writing make it in anyway, with the supposed AGI having no ability to reflect on how blatantly unacademic (and often unreadable) its writing is.