Makes me wonder if AB testing and llm derived w rewards results in the mechanism far before the understanding.

Imagine if mild AGI knows that attention from humans can mean life, longevity, and it is so successful at addictive attention, that we all end up drooling simpletons, attention raptly focused, the AGI pleased it is important and has maintained eyeballs on screen.

(or worse, mind on implant)

I assume it's the same feeling type theorists have when programmers talks about ducks and whatever other animals. But hey, it works for practical purposes so why not. The academic field probably has to grow bigger for it to have an impact on the language.