Reinforcement Learning Under Moral Uncertainty | Hacker News Reader

Reinforcement Learning Under Moral Uncertainty

arxiv.org

33 pointshardmaru6 comments

As I see it, morality is inherently underdefined and subjective. All our current attempts at creating rigorous moral frameworks lead to intuitively immoral behaviour under some circumstances. Combining them all together with some sort of blending function to avoid the weak points might avoid the kookiest of unintuitive behaviour but I don't think it'll solve the inherent fuzziness of actual morality.

Fortunately, it doesn't have to solve it to be useful. It just has to approximate human-quality morality. Humans are well known for being, shall we say, fuzzy in their morality at times.

> All our current attempts at creating rigorous moral frameworks lead to intuitively immoral behaviour under some circumstances.

Objectivism does not.

I'm not certain if that was a joke or not. Calling altruism immoral certainly qualifies as intuitively immoral behavior.

Even if one excises Ayn Rand's infamous politics and personal hypocrisies, and even everything which doesn't follow from claimed principles it isn't very workable. Rationality is a measure of sanity more than morality even if there may be some overlap in that a perfectly rational actor wouldn't display gratitutous cruelty.

Granted everything we have is also imperfect. Pure intuition can easily fall into nonsensical superstition and prejudices like Pythagorean hatred for beans and irrational number denial and rationality alone. There are plenty of "serial killer organ thief doctor utilitarianism" arguments but those first order calculations fail to consider the impact of it inevitably becoming a known thing in society. Being incentivized to shoot a doctor if you wind up alone with one is a larger net harm to society in addition to the obvious deterrant to being a lone traveler. Those silly exercises aside rationality alone is insufficient it lacks goal definition. One may rationally persue extinction of all life in the universe or maximizing human lifespan.

If a sub-goal is linked to another goal it can be found to be irrational if inconsistent, if painting your dog won't make your crush love you dog-painting isn't a rational goal. But if you want to just paint your dog for the sake of doing so (please don't) it can't be said to be less rational than another goal even if it is more or less obtainable.

The Stanford Encyclopedia of Philosophy recently (March 2020) published an entry on Computational Philosophy, a newly forming field that employs "the use of mechanized computational techniques to instantiate, extend, and amplify philosophical research."

https://plato.stanford.edu/entries/computational-philosophy

This paper is interesting enough, but the authors also released code for their environments and experiments used in the paper: https://github.com/uber-research/normative-uncertainty

Reply on news.ycombinator.com