The Black Hole Objection to Longtermism and Consequentialism
schwitzsplinters.blogspot.com
schwitzsplinters.blogspot.com
Human beings are known for values that are risk-averse and loss-averse relative to the supposed "rational" positions of optimizing an abstract utility function.
If someone has, say, a plan to murder X many people to produce Y many future, better people, I have no objection to social sanctions such as prison for them. I'm loss averse, I'm not promiscuously maximizing people-goodness and this is a very natural, human reaction.
It's completely reasonable that we will spend far more resources defending the things, the people, the lives and even the pets we have, compared to the resources spent getting hypothetically different versions of these things. Part of it is one never knows if the hypothetical things will be as good and part of it is one never knows if the calculations we make actually will pan out.
And the "longtermists" often multiply together numbers together they have no basis for knowing in the first place (people in black wholes, people in galactic civilizations, etc) - they fail to conceive that some things more unknowable than probable or improbable and that in such situation loss aversion is especially reasonable - despite going against their
1. We don’t know how to do good risk modeling with huge unknowns. How do you know the universe you create will meet the criteria that would put into the benefit part of the cost/benefit curve? So we don’t know X. We don’t know what the magnitude of the benefit is. Is it really 1e9? And we don’t know the probability of failure. Is it really 1e-6? So sure, for some random set of hypothetical numbers, the cost benefit trade off is there. But we don’t know what the real numbers are and we don’t know what the uncertainty space is.
2. This argument implicitly puts cost/benefit into a single linear variable. But we know that’s not the case. What if the harm is an exponential and the probability of it / magnitude has different scaling factors than your benefit function?
3. If all you need is a planet and you have this tech, what is the likelihood that you can’t use a planet other than Earth or you can’t migrate humans away first?
This is like those mathematical logic puzzles that get you to conclude that 1=2 because an interim step was a division by 0.
And thus your stubbornness leads to this kind of nonsense.
Consequentialism, deonthological ethics, ten commandments, civil laws. Your rules will never be complete enough to make 100% accurate decisions.
When will we learn to avoid this kind of nonsense?
Consequentialism leads to splitting hairs as you reach the fuzzy edges (and more often long before that), giving us self-serving philosophies like effective altruism.
We live in a messy world. Any rules we live by need to know and accept their limits.
Ethics, as in, a set of rules to separate the right from the wrong. I'm not aware of any ethical system which does not try to be universal and accurate. And if the world is so complex that you can't describe it in rules, then ethics is impossible.
Needless to say, I disagree with that notion. Not on the grounds that it's an absurd conclusion, but because an ethical framework doesn't need to be complex in principle, and because simple rules can be accurate.
Except the nihilists. They already don't bother with it.
But the postmodernism view of "ethics is incomplete" is an interesting one. Can a society where this is accepted exist and thrive, where two people with identical ethical frameworks can come to different conclusions? Would ethics even be worth having?
I've only seen one situation where ethics stops making sense, but it was falling apart at really far-fetched edges which are not normally possible to encounter: https://www.lesswrong.com/posts/5iZTwGHv2tNfFmeDa/on-infinit...
Our laws are modeled after certain ethical frameworks, but with the tacit understanding that they cannot completely capture morality and justice, and that morality (and thus our ideas of valid ethics and justice) is temporal (i.e. it will naturally drift over time). That's why laws can be challenged and amended (with certain ethical and moral principles that we are pretty confident will never change codified into a constitution).
It's fascinating to watch the moral evolution of humanity over time, seeing which ideas falter and which ones endure. Culture is the one living thing that evolves without dying (well, except in the case of genocide).
I don't even consider it a goal. Ethical rules demonstrably "fail" (as in the attached article) when they don't match someone's intuition about ethics (not about the world; the world at large doesn't care, unless you consider it a spiritual entity). So all ethics has to do is model someone's intuitions. Which is a completely different task than modelling the world.
This is why I am susprised at your line of thinking. There's no nonsense or foolishness in trying to model one's own intuitions about justice, virtue, etc.
Now people will say that the answer to the trolley problem - AI avoids the trolley problem in the first place + you can minimize the total kinetic energy in the system when you simulate forcing into it - is a cheat because humans can’t do it and it doesn’t address the underlying moral question.
Who cares? We know that the trolley problem doesn’t actually appear in reality in the clean way it’s presented anyway. So if the philosopher is cheating with an unrealistic scenario that would never happen, it’s fair for the student to find a solution by cheating in response (ie Kobayashi Maru).
After such an illogical, non sequitur intro, I'm surprised that this is being upvoted.
If I'm an atheist and materialist in my views, then I am not grasped by any larger context in which to evaluate this decision.
That is: why not a nihilistic flip of the switch?
Restated: whatever meaning we do find in life seem theistic to me.