"It's a very misleading statement to describe today's LLMs as "simply predicting the next word" even if it's in part true."
That's exactly what they do. Don't anthropomorphize a computer program.
That's exactly what they do. Don't anthropomorphize a computer program.
I've watched as GPT-4 accurately modeled the over-justification effect as users started promising tipping and then there was a persistent memory added that revealed they weren't collecting the tip and it output complaints that it was hard to stay motivated not being paid.
That's a very nuanced level of simulation for output of anthropomorphic data with huge implications for synthetic data strategies.
:)
Just wait until someone decides to add emotions to these too!
That's an interesting choice, and might leave you confused as to why Anthropic's system message for the SotA model at the moment talks about it being 'happy' to do tasks (a prompt strategy I was mentioning months ago here on HN).
The data is anthropomorphic. We should expect anthropomorphic behavior and modeling from the LLMs if they do a halfway decent job and expect even more of it as they do a better job.