Hacker News Reader

Comment by goldenkey | Hacker News Reader

I for one, would greatly forgo reward if it were to avoid suffering. I don't think intelligence is simplistic enough to be driven by just a scalar field. I think like physics, we'll likely need two scalars, or a vector, or a spinor, etc.

If "-1" is suffering, and "+1" is reward, then they cancel out. In my own experience, suffering and reward do not cancel out. They have to be on different dimensions. What are your thoughts?

View on HN

"Negative utilitarianism is a form of negative consequentialism that can be described as the view that people should minimize the total amount of aggregate suffering, or that they should minimize suffering and then, secondarily, maximize the total amount of happiness. It can be considered as a version of utilitarianism that gives greater priority to reducing suffering (negative utility or 'disutility') than to increasing pleasure (positive utility). This differs from classical utilitarianism, which does not claim that reducing suffering is intrinsically more important than increasing happiness."

https://en.wikipedia.org/wiki/Negative_utilitarianism

In studies it seems to take about 5 positive experiences to undo the mental impact of 1 negative experience. [1]

So yes - generally these are not considered equal. From almost every perspective, this also makes sense - trading 1 for 1 doesn't lead to you improving your situation.

[1] https://www.seekingbalance.com.au/wp-content/uploads/2016/06...

The claim here is even stronger, and somewhat different: that suffering and reward are fundamentally different kind of things, and cannot simply undo each other, not at any exchange rate. The difference is qualitative, not quantitative. For example, if you lose your eyesight, it's such a big loss, that most likely nothing can balance it out ever. It has no "five times as good" equivalent.

This is exactly what I meant. Thank you for providing a much more accurate and elegant description.

High cost of suffering (e.g. being ran over by a car when street crossed on red light) and relatively small reward for corresponding success (meh, just crossed the street in the example) might explain it.

In psychology/behavioral economics this is known as loss aversion. Most people would rather avoid losing $5 than not earning $5. But given the rather broad view of reinforcement learning this paper adopts, I would guess that the authors would not view this difference in sign as important. (And indeed, in the more ML/AI view of RL, viewing reward as a negative loss or cost is quite standard and for the most part the sign is seen as an arbitrary convention).

In RL, reward does not mean positive only. You could train an agent on purely negative rewards and it would optimize for the policy that is least negative.

Also, +1 and -1 do cancel each other out by definition. If your model has suffering greater than succeeding, then you would assign suffering a greater penalty, like -10 vs +1.

This just points to a different way of defining the reward function, no? You would weigh suffering with a high negative weight.

In the simpler case you are optimizing on a single scale between maximum suffering and maximum happiness.

In the proposed, more complex case you have two independent scales, the suffering scale (on which you want to primarily minimize), and the happiness scale (on which you want to secondarily maximize).

Reply on news.ycombinator.com