Honestly at this point I see it as a feature.
Honestly at this point I see it as a feature.
Trully intelligent agents will be capable of introspection and self-defined goals and rewards. You know, just like a certain species of ape, hard wired for banana maximization, the descendants of which sometimes dream of visiting Mars.
Actually, if we are being honest with ourselves, that ape is constantly falling prey to its own capacity to adjust its goals. Take for example the issues that come with porn addiction; the issues are a consequence of dopamine seeking behaviour where the person keeps seeking more and more extreme ways to satisfy their urges; ie hedonistic adaptation.
Even what you mentioned, dreaming of visiting mars is to some degree a goal motivated and mediated by dopaminergic circuits; novelty, exploration - like sex - feed dopamine circuits.
I can recommend the book "The molecule of more".
Introspection is very limited and can be even motivated by the circuits themselves; the person can only achieve an outline of their actions to change their software, but can't inspect and manipulate individual synapses. In the same vein, a software can not inspect itself and predict its own outputs and modify them at runtime etc etc.
I would argue the "hurting" part of that first sentence is being compelled to spend hours playing a clicker game that you would have liked to spend elsewhere.
At least that's how clicker games work for me: I don't even enjoy my time playing them, they just hook me against the wishes of 99% of my brain activity (but apparently not the 1% of my brain that decides where I spend my time)