Hacker News Reader

> I derive a lot of my personal sense of worth from what I do - from what I build and for what I contribute to other people. I don't know what I would do in a world where that was no longer needed. Is that healthy? Probably not. (It's actually something I'm working through at the moment, that I increasingly understand as a consequence of both personal and societal-level abuse.)

Since you are wondering about the healthiness of that emotion, try to read C.S. Lewis "The Four Loves": the Need love/Gift love opposition may have answers for you.

> If tomorrow the greatest dream of AI was achieved, and we had a perfect benevolent AI that would shepherd humanity to comfort and safety forever, that would be just about the best outcome imaginable in material terms. But I think I'd be unhappy with it, because I derive a lot of my personal sense of worth from what I do - from what I build and for what I contribute to other people.

But surely the Shepherd would be aware of your soul's needs as well, and could pretend that your help is required somewhere in the system.

Perhaps so! I thought about writing a story of an AI like that - something that cures diseases and prevents war and gives you all the tools to be smarter and healthier and more virtuous, but otherwise says "nope, humanity, you gotta figure this out yourself if you want the reward". It'd be an interesting sci-fi setting.

Many books in Iain M Banks’ Culture series describe how humans find meaning or otherwise entertain themselves in an AI-led post-scarcity utopia

But surely the Shepherd would be aware of your soul's needs as well, and could pretend that your help is required somewhere in the system.

I think most people wouldn’t be fooled by this. People intuitively understand when their work is superfluous. They see that even if they slack off nothing blows up, no alarm bells sound, no production lines grind to a halt, no one dies.

> that would shepherd humanity to comfort and safety forever

If Calhoun's mouse utopia experiments are anything to go by, such a seemingly idyllic situation would spell out humanity's death in very real terms.

Calhoun's utopia was about overpopulation, not about idyllic situations.