Hacker News Reader

Comment by logicchains | Hacker News Reader

Given the recent push towards formally verifying mathematics, it's surprising how comparatively little effort's been put into formalising ethics. Fundamentally an ethical system is not that different from a mathematical one: there are some assumptions, some deductive rules, and a potentially infinite set of conclusions that result from applying those deductive rules to those assumptions. One particularly important property to verify about such a system is consistency: if it can prove conclusion "X", it should not also be able to prove conclusion "not X". An inconsistent system can prove anything, so is essentially useless for reasoning.

View on HN

Philosophers have been trying and failing for millennia to come up with hard rules for ethics. A rule that is essential in one context invariably leads to absurdity in others (there are many examples involving people strapped to train tracks). So an attempt at formalizing this will not just be ambiguous, but likely unsound and contradictory. While mathematics seem to be this perfect structure encoded in the fabric of the universe, just waiting for us to discover and describe it, ethics in contrast seems to be a very fuzzy, imperfect, ad-hoc creation of mankind.

>Philosophers have been trying and failing for millennia to come up with hard rules for ethics.

They've been trying to find one true/correct system of ethics, which is doomed to fail due to https://en.wikipedia.org/wiki/Regress_argument. What I'm talking about is not trying to find new systems, rather just formalising existing systems, so that they can be analysed more rigorously and checked for consistency. Much as for instance one can formalise and compare homotopy type theory and set theory as foundations for real analysis.

Rudolf Carnap worked hard on formalizing ethics (and everything else) in the 20th century; the project was a total flop.

It turns out you can't formalize anything that doesn't have proofs -- in other words, it basically only works for what we'd call "mathematics" or "pure logic". You can't formalize literally anything else in philosophy, at all.

The regress argument has nothing to do with ethics in particular. It applies just as much to any non-proof area of study, including all of science, history, and even ordinary observations and recollections.

>It turns out you can't formalize anything that doesn't have proofs -- in other words, it basically only works for what we'd call "mathematics" or "pure logic". You can't formalize literally anything else in philosophy, at all.

What do you mean by "doesn't have proofs" here? It's demonstrably possible to develop a formalised philosophy: see Spinoza's Ethics for an example. It's impossible to prove that one formalisation is more "correct", but the same applies to mathematics, via Gödel's second incompleteness theorem.

>The regress argument has nothing to do with ethics in particular. It applies just as much to any non-proof area of study, including all of science, history, and even ordinary observations and recollections.

It also applies to maths in the form that a system cannot prove its own consistency. Yet that doesn't stop us formally analysing, comparing and making use of different mathematical systems.

> It turns out you can't formalize anything that doesn't have proofs -- in other words, it basically only works for what we'd call "mathematics" or "pure logic". You can't formalize literally anything else in philosophy, at all.

This couldn’t be further from the truth. There is a wealth of logics that have been developed and studied within philosophy, that aim to formalize various philosophical concepts. See modal logics, deontic logics, logics about knowledge and belief, and so on.

The notion of a "proof" is only meaningful in a formal context.

Logic, mathematics, computer science, proof theory... same thing. https://en.wikipedia.org/wiki/Proof_theory

We are eternally stuck interpreting all of our theories in the context of their meta-theoretic assumptions.

Not sure we’re talking about the same thing here. I was replying to your claim that nothing in philosophy (apart from mathematics) can be formalized. The role of logic is not to solve the metaphysical problem of infinite regress.

Logic itself is a formalism.

Subject to all the constraints of formal language theory/linguistics.

It fails to address the symbol-grounding problem.

https://en.wikipedia.org/wiki/Symbol_grounding_problem

Yeah, ethics is subjective and it's fascinating that many smart people don't realize this. Ultimately it always boils down to "I feel this is right".

Of course, you can have objective claims such as "if all people followed this principle, it would have this outcome".

There is a tension between relative/subjective ethics and absolute/objective ethics. Philosophers seek to discover the absolute, but they cannot shake the human condition

>Of course, you can have objective claims such as "if all people followed this principle, it would have this outcome".

Good luck proving the outcome for anything less than very trivial cases...

Well obviously you can't prove the outcome of making a change in a complex physical system with many biological components. We can barely model a protein.

There has been at least since 1951 since the famous publication of Arrow's impossibility theorem.

Extract from Wikipedia :

"The framework for Arrow's theorem assumes that we need to extract a preference order on a given set of options (outcomes). Each individual in the society (or equivalently, each decision criterion) gives a particular order of preferences on the set of outcomes. We are searching for a ranked voting electoral system, called a social welfare function (preference aggregation rule), which transforms the set of preferences (profile of preferences) into a single global societal preference order. Arrow's theorem says that if the decision-making body has at least two members and at least three options to decide among, then it is impossible to design a social welfare function that satisfies all these conditions (assumed to be a reasonable requirement of a fair electoral system) at once: [...]"

https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theore...

>Fundamentally an ethical system is not that different from a mathematical one: there are some assumptions, some deductive rules, and a potentially infinite set of conclusions that result from applying those deductive rules to those assumptions.

Even simple enough math starting from formal axioms are incomplete (Godel) [1]. Real world ethics, starting by ethical assumptions will be 1000 times as full of holes and inconsistencies. And that's before we even consider language ambiguities in the much more complex "ethical assumptions".

[1] "The first incompleteness theorem states that no consistent system of axioms whose theorems can be listed by an effective procedure (i.e., an algorithm) is capable of proving all truths about the arithmetic of natural numbers. For any such consistent formal system, there will always be statements about natural numbers that are true, but that are unprovable within the system. The second incompleteness theorem, an extension of the first, shows that the system cannot demonstrate its own consistency".

>Even simple enough math starting from formal axioms are incomplete (Godel) [1]. Real world ethics, starting by ethical assumptions will be 1000 times as full of holes and inconsistencies.

Just because a mathematical system like set theory is incomplete (can't prove its own consistency), that doesn't mean it's "full of holes and inconsistencies". And just because a system cannot prove itself consistent, doesn't mean it can't be proven consistent in some larger system (for instance how we can use ZFC to prove that Peano arithmetic is consistent).

If we want to be very reductive about it there is not a great reason to believe we can use formal logic to inform the chosen(?) actions of huge aggregates of molecules jiggling around. All ethical systems that have withstood the test of time (the gold & silver rules) use subjectivity. Everything else falls prey to absurdity.

I recall an objection was raised (possibly by Alasdair MacIntyre? it's been a while since I got wrapped up in all this) that any system of ethics must be inherently self-contradictory. The reason for this is something like any time we spend in pursuit of correcting or preventing some moral wrong is time not given doing the same for another. And there are certainly a great many moral wrongs going on according to any given system of ethics - probably all of equal importance.

Separately we might also ask whether we even want to solve the problem of ethics. Imagine tomorrow we develop an ironclad formal moral system - everyone is forced to agree it is correct through sheer reason alone, and it meaningfully informs our every action. Would you really want to live in such a world? It seems a world without freedom, or one which would quickly become that way. All-encompassing totalitarianism. [0]

In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]

[0] https://www.existentialcomics.com/comic/139

[1] https://ase.tufts.edu/cogstud/dennett/papers/chmess.pdf

>Separately we might also ask whether we even want to solve the problem of ethics.

The article's author presents a good example: for AI ethics. If we want to prove "an AI following this system won't kill all humans (or has e.g. less than 0.1% chance of killing all humans)", we need to be able to formally reason about that system.

>In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]

By that logic, why bother with any of the higher, more abstract fields of mathematics?

> The article's author presents a good example: for AI ethics

You don't need ethics for that, this problem is basically verifying that the system works according to the specification and that it didn't have bugs.

You do for an AGI, because human-level intelligence is too complex to describe in a specification.

There are two issues here though. Firstly, what is the right thing to specify? (This is an ethical proposition, even for fairly simple agents.) Secondly, how can this specification be reliably encoded in the AI's world model? (This gets technically more more challenging the more complex the agent is.)

Have you heard of paraconsistent logic?

Ethics does not aim to practically tell us what we should do at all times, but to give us the tools to think about such question. A system of ethics where two opposing conclusions can be reach could be perfectly fine.

In paraconsistent logics one cannot use a proof of "X" and "not X" to prove any arbitrary statement, so it can indeed still be useful. However we still need to formally reason about the system to be able to establish that it is indeed a paraconsistent logic (that the principle of explosion doesn't apply).

There has actually been quite some work since the 50ies up to the present day on deontic logic, which is reasoning about ethical concepts such as obligation and permission.

There has always been a ton of effort at formalising ethics, political theories and philosophy generally in a mathematical-like way. The whole of modern philosophy, especially political philosophy , is a product of that effort. That's basically what the enlightenment was.

The problem, IMO, is that formalistic, math-like approach produces "an inconsistent system can prove anything, so is essentially useless for reasoning." Going for fundamental, objective truths with a mathematical standard as an ideal can go one of three ways: delusion, nihilism or sociopathy.

Peter Singer, Jeremy Bentham and other utilitarians came up with a "moral math" system that (a) sounded morally reasonable at first pass and (b) are theoretical applicable widely. The problem they encounter is extreme conclusions. Bentham was responsible for some horrific prison/reformatory systems. Singer ends up with some sociopathic conclusions like morally good infanticide.

Ayn Rand went down a philosophical worm-whole trying to tie her philosophical system into one "theory-of-everything" bow where pure (Aristotelian) logic yields conclusive answers to all epistemological, moral, political & economic questions. This certainty, rather than just the extreme or counterintuitive (selfishness as a virtue) elements of objectivism is what gives it that cult-like flavour.

Ayn Rand was simply following the frame created by Marxism, and Trotsky particularly. Marxism of that era was all about airtight rationality. Communism wasn't a choice, but a dictate of pure reason. Marxists got this idea from French revolutionary philosophers, who started with the idea of "rationalising" everything with math-like systems of thought.

What all these have in common is a desire to create irreproachable systems. Like math, like physics. In practice, they created religions. Religions are irreproachable too. The way religions become irreproachable is by banishing heretics. That's what the "secular" rationalist philosophers did to. Rand was famous for her banishments.

If you are after scientifically valid, reductive, Popperian insights only... you might just have to accept that this is an amoral viewpoint. Morals just don't exist in the mathematical realm.

>What all these have in common is a desire to create irreproachable systems. Like math, like physics. In practice, they created religions. Religions are irreproachable too. The way religions become irreproachable is by banishing heretics. That's what the "secular" rationalist philosophers did to. Rand was famous for her banishments.

I'm not talking about using mathematical, logical approach to find the "most correct" philosophy. As Wittegenstein noted in Tractatus, the "most correct" approach is: "To say nothing except what can be said, i.e. the propositions of natural science, i.e. something that has nothing to do with philosophy: and then always, when someone else wished to say something metaphysical, to demonstrate to him that he had given no meaning to certain signs in his propositions".

Rather I'm talking about using formal methods to analyse existing systems. Identify their axioms, identify the deductive rules they permit, and determine how consistent they are. Maybe even formally verify this in a proof assistant. Then it'd be also possible to e.g. explore what different systems can and cannot prove (somewhat akin to https://en.wikipedia.org/wiki/Reverse_mathematics). An example kinda along these lines is Lawvere's work on modelling Hegelian philosophy as modal type theory: https://ncatlab.org/nlab/show/Hegel%27s+%22Logic%22+as+Modal....

I think this take is pretty much what enlightenment and post enlightenment philosophy attempted to do. Benthamites like Singer (and Bentham) do this quite literally: Identifying axioms (the utilitarian principle), deduce principles (utilitarian math) and strive (or argue for) consistency.

Utilitarians tended to end up with some pretty abhorrent extremities, by faithfully following their principles to some pretty weird places. If you want to be strict in your formalism, this means utilitarians can either accept stuff like Singer's infanticide or involuntary euthanasia arguments or they can reject the underlying principles.

More commonly (certainly the case for Rand, Trotsky and most other modernists), they go down a rabbit-hole of increasingly complicated fundamentals... The fundamentals (principles/axioms) need tweaking in order to support the complex conclusions. Most modernists wanted moral theories that also tied in neatly with epistemology, political philosophy and often economics. Being modernists, and committed to deductive reasoning, they wanted a tight basket of fundamentals to yield the whole thing. Things tend to get complicated.

I like your quote. Very apt. That period was, IMO, the last (and probably best) attempt at modernist epistemology. Trotski, Rand, Popper, Wittegenstein...

My 100-years later thoughts on that is that (a) Popper was right and (b) Popper (ironically) ended modernist, purely rational moral & political philosophy. Drwaing that hard line between scientific reasoning and the rest of philosophy was essential. But, it didn't change the fact that science (to reverse your quote) has nothing to say about philosophy.

Leftist philosophy made a U-turn following this period. Trotski was the last rationalist here, and subjective philosophies dominated self described leftist thought from then on. Rand's philosophy was de-rationalised and de-modernised before becoming the operative political philosophy she wanted. Popper's own collaborators (famously) Hayek were careful about Popper's lines and avoiding "pseudoscience," but their philosophies continued on to form the basis of post-Popperian pseudoscience. Pseudoscience in all but the most technical sense.

...Basically, postmodernism happened for a reason. Modernist thought kept hitting brick walls. IMO, these were the "phenomenon of a null result." Morals, politics and such just don't have a purely rational basis. No philosophy is both perfectly consistent and parsimonious with our moral senses.

At first pass, these conclusions lend to nihilism. But this is only true to the extent that we stay within the modernist frame. Morals either exist, and can be analysed in a strictly logical way, or they don't. It ignores the fact that we do, in actual reality, have morals.

This is what happens when you get too attached to a philosophical frame. The "works in practice, but not in theory" problem.

Analysing philosophies (especially enlightenment oriented ones) in this manner is kinda what most college courses do. It's getting tired though, and less popular. IMO, an honest take on this would be a "why every philosophy is wrong" course... but this gets trite fast too. If you set out to identify inconsistencies, it isn't hard. Once you get into defences, "why the philosophy isn't actually flawed..." it descends into something like high medieval philosophy. Boring and anachronistic stuff like "the problem of evil" but without god.

Problems in philosophy rarely get solved. We just move on eventually.

>Problems in philosophy rarely get solved. We just move on eventually.

We've now got a greater motivation to solve them than before: so we can answer the question "will an AI following this set of beliefs eventually end up killing us all?"

That's a different sort of problem, I think. It doevetails with a lot of "two rail carts heading toward death" scenarios beloved by moral philosophy, but I still don't think it's the same problem.

Moral philosophy ultimately deals with what morals are. But, because morals predate moral philosophy, the moral philosophy isn't free to come to whatever conclusions it comes to. If the philosophy doesn't agree with what we already think is moral (beyond helping us navigate ambiguities), we just reject it.

If we want to give AI a Asimovian ethic, we're just concerned with making it do what we want it to do.

>If we want to give AI a Asimovian ethic, we're just concerned with making it do what we want it to do.

Isn't that the whole point of most "safe AGI" research? Not that it's necessarily a good thing, but I imagine that if there was a chance that the most ethical option was in fact to destroy all humans, most people would prefer AI to have "do what we want it do" ethics than "do what's most ethical" ethics.

>>Isn't that the whole point of most "safe AGI" research

IDK, honestly. Not my lane. At an uneducated guess, I would say that it's probably a more common topic of "safe AGI" press articles than actual research. Dunno though.

It's too theoretical. When technology is theoretical, we tend to personify it. If we imagined an automated hotel in the 60s, we'd probably imagine a humanoid robot standing at reception. In the 2000s, that image would have been a screen at reception. Today, it's an app that also unlocks the door and there is no lobby.

My guess is that by the time AI can be programmed via abstract ethics, it will have upended so much that its just hard to reason about from our current position.

I hope "AI ethics" are focused on stuff that's applicable now.

Reply on news.ycombinator.com