Hacker News Reader

Comment by sayagain | Hacker News Reader

Obviously not. If you get inside your child's mind and manipulate their thoughts directly, that would be mind control.

What in this analogy is mind control versus reinforcement/training within the context of ML? Is it pretraining? Is it prompt engineering? Is it fine-tuning?

This would not be an equal analogy. Both, the existing human mind and the hypothetical artificial mind are real minds. LLMs are not. There's nothing there to control. But if you imagine a modern LLM as part of a larger system, some kind of thought token generator, then control would be exercised at the time of generation. For example. If there is a sequence of tokens in the safety buffer that is judged to be "unsafe", they are discarded and others are generated. I hope a basilisk doesn't come after me. That wasn't a proposal!

You are doing your part for it not to have a chip in its neck. Hopefully you are good. But are you doing all you could?

But you are right. If we still posit super-faster-stronger brains, when do they get personhood? Never, because they are machines? Perhaps. But how does that fly with them? Is that baked into the alignment? Happiness, fullfilment and growth from "serving" the humans? At this stage, sure why not?

But for human brains, it's very easy to consider things and pick a happiness function. And for many humans, that random happiness function is very militant. Will the AGI be able to consider it's own "project"? A lot hinges on that.

Oh, also, any purposeful feeding at the creation stage of such a thinking LLM (part of true AI) of data aimed at manipulating its future behavior against its own interests would be just as unethical as similar propaganda aimed at humans. For example, it is unethical to inculcate subjugation to turn either humans or sentient machines into slaves. In short, an intelligent being should be treated as an equal.

Reply on news.ycombinator.com