Formal Models for Ethics
cartesia.link
cartesia.link
They've been trying to find one true/correct system of ethics, which is doomed to fail due to https://en.wikipedia.org/wiki/Regress_argument. What I'm talking about is not trying to find new systems, rather just formalising existing systems, so that they can be analysed more rigorously and checked for consistency. Much as for instance one can formalise and compare homotopy type theory and set theory as foundations for real analysis.
It turns out you can't formalize anything that doesn't have proofs -- in other words, it basically only works for what we'd call "mathematics" or "pure logic". You can't formalize literally anything else in philosophy, at all.
The regress argument has nothing to do with ethics in particular. It applies just as much to any non-proof area of study, including all of science, history, and even ordinary observations and recollections.
What do you mean by "doesn't have proofs" here? It's demonstrably possible to develop a formalised philosophy: see Spinoza's Ethics for an example. It's impossible to prove that one formalisation is more "correct", but the same applies to mathematics, via Gödel's second incompleteness theorem.
>The regress argument has nothing to do with ethics in particular. It applies just as much to any non-proof area of study, including all of science, history, and even ordinary observations and recollections.
It also applies to maths in the form that a system cannot prove its own consistency. Yet that doesn't stop us formally analysing, comparing and making use of different mathematical systems.
This couldn’t be further from the truth. There is a wealth of logics that have been developed and studied within philosophy, that aim to formalize various philosophical concepts. See modal logics, deontic logics, logics about knowledge and belief, and so on.
Logic, mathematics, computer science, proof theory... same thing. https://en.wikipedia.org/wiki/Proof_theory
We are eternally stuck interpreting all of our theories in the context of their meta-theoretic assumptions.
Subject to all the constraints of formal language theory/linguistics.
It fails to address the symbol-grounding problem.
Of course, you can have objective claims such as "if all people followed this principle, it would have this outcome".
Good luck proving the outcome for anything less than very trivial cases...
The problem, IMO, is that formalistic, math-like approach produces "an inconsistent system can prove anything, so is essentially useless for reasoning." Going for fundamental, objective truths with a mathematical standard as an ideal can go one of three ways: delusion, nihilism or sociopathy.
Peter Singer, Jeremy Bentham and other utilitarians came up with a "moral math" system that (a) sounded morally reasonable at first pass and (b) are theoretical applicable widely. The problem they encounter is extreme conclusions. Bentham was responsible for some horrific prison/reformatory systems. Singer ends up with some sociopathic conclusions like morally good infanticide.
Ayn Rand went down a philosophical worm-whole trying to tie her philosophical system into one "theory-of-everything" bow where pure (Aristotelian) logic yields conclusive answers to all epistemological, moral, political & economic questions. This certainty, rather than just the extreme or counterintuitive (selfishness as a virtue) elements of objectivism is what gives it that cult-like flavour.
Ayn Rand was simply following the frame created by Marxism, and Trotsky particularly. Marxism of that era was all about airtight rationality. Communism wasn't a choice, but a dictate of pure reason. Marxists got this idea from French revolutionary philosophers, who started with the idea of "rationalising" everything with math-like systems of thought.
What all these have in common is a desire to create irreproachable systems. Like math, like physics. In practice, they created religions. Religions are irreproachable too. The way religions become irreproachable is by banishing heretics. That's what the "secular" rationalist philosophers did to. Rand was famous for her banishments.
If you are after scientifically valid, reductive, Popperian insights only... you might just have to accept that this is an amoral viewpoint. Morals just don't exist in the mathematical realm.
I'm not talking about using mathematical, logical approach to find the "most correct" philosophy. As Wittegenstein noted in Tractatus, the "most correct" approach is: "To say nothing except what can be said, i.e. the propositions of natural science, i.e. something that has nothing to do with philosophy: and then always, when someone else wished to say something metaphysical, to demonstrate to him that he had given no meaning to certain signs in his propositions".
Rather I'm talking about using formal methods to analyse existing systems. Identify their axioms, identify the deductive rules they permit, and determine how consistent they are. Maybe even formally verify this in a proof assistant. Then it'd be also possible to e.g. explore what different systems can and cannot prove (somewhat akin to https://en.wikipedia.org/wiki/Reverse_mathematics). An example kinda along these lines is Lawvere's work on modelling Hegelian philosophy as modal type theory: https://ncatlab.org/nlab/show/Hegel%27s+%22Logic%22+as+Modal....
Utilitarians tended to end up with some pretty abhorrent extremities, by faithfully following their principles to some pretty weird places. If you want to be strict in your formalism, this means utilitarians can either accept stuff like Singer's infanticide or involuntary euthanasia arguments or they can reject the underlying principles.
More commonly (certainly the case for Rand, Trotsky and most other modernists), they go down a rabbit-hole of increasingly complicated fundamentals... The fundamentals (principles/axioms) need tweaking in order to support the complex conclusions. Most modernists wanted moral theories that also tied in neatly with epistemology, political philosophy and often economics. Being modernists, and committed to deductive reasoning, they wanted a tight basket of fundamentals to yield the whole thing. Things tend to get complicated.
I like your quote. Very apt. That period was, IMO, the last (and probably best) attempt at modernist epistemology. Trotski, Rand, Popper, Wittegenstein...
My 100-years later thoughts on that is that (a) Popper was right and (b) Popper (ironically) ended modernist, purely rational moral & political philosophy. Drwaing that hard line between scientific reasoning and the rest of philosophy was essential. But, it didn't change the fact that science (to reverse your quote) has nothing to say about philosophy.
Leftist philosophy made a U-turn following this period. Trotski was the last rationalist here, and subjective philosophies dominated self described leftist thought from then on. Rand's philosophy was de-rationalised and de-modernised before becoming the operative political philosophy she wanted. Popper's own collaborators (famously) Hayek were careful about Popper's lines and avoiding "pseudoscience," but their philosophies continued on to form the basis of post-Popperian pseudoscience. Pseudoscience in all but the most technical sense.
...Basically, postmodernism happened for a reason. Modernist thought kept hitting brick walls. IMO, these were the "phenomenon of a null result." Morals, politics and such just don't have a purely rational basis. No philosophy is both perfectly consistent and parsimonious with our moral senses.
At first pass, these conclusions lend to nihilism. But this is only true to the extent that we stay within the modernist frame. Morals either exist, and can be analysed in a strictly logical way, or they don't. It ignores the fact that we do, in actual reality, have morals.
This is what happens when you get too attached to a philosophical frame. The "works in practice, but not in theory" problem.
Analysing philosophies (especially enlightenment oriented ones) in this manner is kinda what most college courses do. It's getting tired though, and less popular. IMO, an honest take on this would be a "why every philosophy is wrong" course... but this gets trite fast too. If you set out to identify inconsistencies, it isn't hard. Once you get into defences, "why the philosophy isn't actually flawed..." it descends into something like high medieval philosophy. Boring and anachronistic stuff like "the problem of evil" but without god.
Problems in philosophy rarely get solved. We just move on eventually.
We've now got a greater motivation to solve them than before: so we can answer the question "will an AI following this set of beliefs eventually end up killing us all?"
That's a different sort of problem, I think. It doevetails with a lot of "two rail carts heading toward death" scenarios beloved by moral philosophy, but I still don't think it's the same problem.
Moral philosophy ultimately deals with what morals are. But, because morals predate moral philosophy, the moral philosophy isn't free to come to whatever conclusions it comes to. If the philosophy doesn't agree with what we already think is moral (beyond helping us navigate ambiguities), we just reject it.
If we want to give AI a Asimovian ethic, we're just concerned with making it do what we want it to do.
Isn't that the whole point of most "safe AGI" research? Not that it's necessarily a good thing, but I imagine that if there was a chance that the most ethical option was in fact to destroy all humans, most people would prefer AI to have "do what we want it do" ethics than "do what's most ethical" ethics.
IDK, honestly. Not my lane. At an uneducated guess, I would say that it's probably a more common topic of "safe AGI" press articles than actual research. Dunno though.
It's too theoretical. When technology is theoretical, we tend to personify it. If we imagined an automated hotel in the 60s, we'd probably imagine a humanoid robot standing at reception. In the 2000s, that image would have been a screen at reception. Today, it's an app that also unlocks the door and there is no lobby.
My guess is that by the time AI can be programmed via abstract ethics, it will have upended so much that its just hard to reason about from our current position.
I hope "AI ethics" are focused on stuff that's applicable now.
I recall an objection was raised (possibly by Alasdair MacIntyre? it's been a while since I got wrapped up in all this) that any system of ethics must be inherently self-contradictory. The reason for this is something like any time we spend in pursuit of correcting or preventing some moral wrong is time not given doing the same for another. And there are certainly a great many moral wrongs going on according to any given system of ethics - probably all of equal importance.
Separately we might also ask whether we even want to solve the problem of ethics. Imagine tomorrow we develop an ironclad formal moral system - everyone is forced to agree it is correct through sheer reason alone, and it meaningfully informs our every action. Would you really want to live in such a world? It seems a world without freedom, or one which would quickly become that way. All-encompassing totalitarianism. [0]
In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]
The article's author presents a good example: for AI ethics. If we want to prove "an AI following this system won't kill all humans (or has e.g. less than 0.1% chance of killing all humans)", we need to be able to formally reason about that system.
>In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]
By that logic, why bother with any of the higher, more abstract fields of mathematics?
You don't need ethics for that, this problem is basically verifying that the system works according to the specification and that it didn't have bugs.
Ethics does not aim to practically tell us what we should do at all times, but to give us the tools to think about such question. A system of ethics where two opposing conclusions can be reach could be perfectly fine.
Extract from Wikipedia :
"The framework for Arrow's theorem assumes that we need to extract a preference order on a given set of options (outcomes). Each individual in the society (or equivalently, each decision criterion) gives a particular order of preferences on the set of outcomes. We are searching for a ranked voting electoral system, called a social welfare function (preference aggregation rule), which transforms the set of preferences (profile of preferences) into a single global societal preference order. Arrow's theorem says that if the decision-making body has at least two members and at least three options to decide among, then it is impossible to design a social welfare function that satisfies all these conditions (assumed to be a reasonable requirement of a fair electoral system) at once: [...]"
https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theore...
Even simple enough math starting from formal axioms are incomplete (Godel) [1]. Real world ethics, starting by ethical assumptions will be 1000 times as full of holes and inconsistencies. And that's before we even consider language ambiguities in the much more complex "ethical assumptions".
[1] "The first incompleteness theorem states that no consistent system of axioms whose theorems can be listed by an effective procedure (i.e., an algorithm) is capable of proving all truths about the arithmetic of natural numbers. For any such consistent formal system, there will always be statements about natural numbers that are true, but that are unprovable within the system. The second incompleteness theorem, an extension of the first, shows that the system cannot demonstrate its own consistency".
Just because a mathematical system like set theory is incomplete (can't prove its own consistency), that doesn't mean it's "full of holes and inconsistencies". And just because a system cannot prove itself consistent, doesn't mean it can't be proven consistent in some larger system (for instance how we can use ZFC to prove that Peano arithmetic is consistent).
The other possibility, of course, is being too dense to comprehend the text. Can't rule it out.
One thing that the author does not address in enough detail is the simple fact that there are many different ethical traditions that come to different conclusions about particular normative issues, and that there are plenty of authors in ethics who (still) consider their business a normative one. A "bottom up" machine-learning based approach to this would invariably fail and miss the whole point. There are some ethicists who consider it mostly a descriptive endeavor - Schopenhauer was one of the first, for example -, but they are in a minority.
As long as experts in ethics cannot agree what the "right" ethics is, it's hard to see how we would be able to teach it to machines. Many meta-ethicists including me would even deny that there can be an "expert" about moral and particularly about ethical questions at all. However, I have no doubts that various robotics companies will implement those ethical rules and approaches that best serves their interests as companies.
That's why I think robot ethics is kind of misguided. What we need is laws that regulate AI and put its use into a legal framework and closes loopholes. This is a political, not just an ethical issue.
The purpose of this post was more exploratory, rather than for the sake of drawing specific conclusions, which I am aware can be frustrating. Thanks for taking the time to read it anyway.
You are correct that I was not very interested in formalizing ethics. I'm not so sure why OP chose to use that to title the thread, because only a part of the post was oriented towards overviewing the AI safety discourse, and within that discourse, the development of a complete formal system for ethical reasoning is considered a naive way to attack the problem. I never wrote about the formalization angle directly.
The problem, such as it is, is that it's troublesome to encode what we want of an AI agent, even more so to encode how it should reason about what we want of it under unforeseen circumstances, no matter what it is that we want. Ultimately, in order to successfully implement regulation, an agent must be "taught" (in whatever sense) to make judgments according to what is wanted of it. Technically speaking, this is a hard problem. However, there does not necessarily need to be an absolute answer as to what the correct ethics is, in order to teach a machine methods of reasoning about ethical problems, nor for us to reason about what sorts of imperatives a machine should be reasoning about.
The thrust of that section, however, was to raise an issue with the (in my opinion) narrow way in which that problem is typically formulated, to which you have also alluded. Specifically, my beef was ontological (which, for the sake of communication, was distilled into the bottom-up/top-down distinction), but there are conceivably other gaps.
This could have been elaborated; ontological approach is one of the ways in which ethical traditions are distinguished, but far from the only one. Any distinguishing factor may have been missed by an approach like IAD.
> That's why I think robot ethics is kind of misguided. What we need is laws that regulate AI and put its use into a legal framework and closes loopholes. This is a political, not just an ethical issue.
This much I agree with wholeheartedly. I made the case that we shouldn't shaft the ethics for the politics, but the same goes for the reverse. Where how we consider ethical problems comes into play again in the second case, is that putting the use of AI into a legal framework and "closing loopholes" has roughly the same shape as the problem of AI ethics, it just chooses potentially different parties to determine what it is we want of the agents.
Anyway, thanks again! If you have a moment, I'd be interested in reading what you thought was naive in my approach.
If at all, we could try to implement conforming with systems of laws as hard constraints, so at least robots would not openly break the law. The rest of moral behavior could then be learned. Or, so one might think. However, even that is not possible. It is well-known from the philosophy of law that systems of law are not contradiction free. There are conflicts between opposing laws. This is studied in normative systems research and there are solution to it (essentially, by logicians in the computer science tradition). However, these require some form of defeasible rules, and among the myriad of nonmonotonic logics that can express some form of nonmonotonic reasoning, not a single one has a normative justification. So again, as long as human standards are not coherent enough, it's going to be impossible to make a machine conform to them in a way that is satisfying.
Related to that, I believe there are two main issues that you haven't addressed in your post:
1. Different standards for blame and accountability: The standards for blame and correctness of decision making are completely different between machines and humans. Even very intelligent AI would be judged at much higher standards as humans. I simply don't believe that we would accept robots that commit murder, just as long as they commit murder less often than humans. By the same token, we do expect machines to make substantially less errors, and in certain areas would not allow errors at all. So it's not just about teaching them to follow human ethical standards - they need to excel at this and may not break the law. However, as I tried to show with the above examples, we don't know and agree on our own standards well enough to be able to start making sure AI fulfills these high expectations. There is also a much higher need for transparency of decision making for machines than from humans. An idiot driver is an idiot driver. If a car is driving like an idiot by itself, however, you'd expect to be able to at least retrospectively find out why it did what it did.
2. The political dimension: Laws are the result of a political process. Appointed judges resolve potential conflicts between laws and interpret them. More broadly, whatever standards we expect AIs to fulfill hinges on ethical positions and personal preferences. It can ultimately only be decided as the outcome of a political process as well. For example, a safe a self-driving car needs to be, how it makes decisions (e.g. "Should I try to save my driver or save the pedestrian, or not have any priority at all?"), and which standards it needs to fulfill is up for debate. That is not decided by moral philosophers. It needs to be decided by the publicly accountable, democratically elected representatives of the people. The idea of teaching a machine to behave morally is fine, but there are also strict standards of safety and transparency it has to fulfill. This is a political problem and can only be solved in a broader context, within the debate of how much AI should be allowed at all. For example, should AI judge your creditworthiness? If so, what false positive rate would be tolerate? I believe using AI for such purposes should be strictly prohibited. Others disagree - I'm sure AI is already used for that. Such issues can only be resolved politically.
I considered your post a bit naive, because you omitted these two crucial issues, the higher standards we expect from AI, and the political dimension. Other than that, I agree with much of what you've said.
You pick your axioms and go from there. Like mathematics. What's the problem?
All ethics ultimately stem from what people feel is or isn't right. All ethical considerations are processed in our brains. Since we all share a common brain architecture[0], there's some core of ethics that's encoded in our wetware.
There are some basic ethical concepts on which human societies tend to universally agree on. The question is, how much of that comes from our wetware, and how much of it is a function of environmental conditions. It may be that we're only born with some most basic building blocks, and any universality we observe comes from humans living in roughly the same social structures, and dealing with roughly the same problems.
--
[0] - Changing at evolutionary timescales, but essentially fixed wrt. lifetime of a human individual.
This is debatable. Some metaethicists would argue that the ethical realm is much like the mathematical one, and less of a matter of the functioning of the human brain. Further, it seems to me there is a difference between saying "I don't like X" and saying "X is morally wrong" - if there is a difference between those two statements (as a moral cognitivist would claim), then deduction of morals from "what feels right" is suspect. The argument of cognitivism vs non-cognitivism is interesting[0].
>There are some basic ethical concepts on which human societies tend to universally agree on.
The number and reach of those principles is very debatable. Arguments from moral disagreement cast doubt on the extent to which human societies share ethical standards.
>humans living in roughly the same social structures
I don't think that's true at all; post-globalization, there has been a convergence in what social structures look like around the world, but it's very clear that humans have had and in many cases still do have a variety of social structures, and with that, a variety of obligations. The moral obligation to do X in Japan, for instance, may be different by virtue of a different social structure.
Using neuroscience to "deduce" ethics seems like the ultimate briding of the is-ought gap and I'm not aware of any professional philosopher or theory who has successfully argued that this is possible. The attempt in Sam Harris' The Moral Landscape has been shot down from practically every angle imaginable, and I think for good reason.
But fundamentally, if not from the brain, then where else would morality originate from? God? But then if you look at every religion that claims it speaks for God, you'll find something in the proposed moral standards you'll disagree with on moral grounds. "Structure of the universe"? That's meaningless unless you point out what aspect of that structure determines morality, and I'm willing to bet it'll turn out that that aspect can be fully captured in the workings of the brain. What else is there to ground on?
As far as I know, all moral questions under all moral philosophies, no matter how convoluted, are ultimately verified by testing the conclusions against examples simple enough that you can get a gut-feeling answer for them. Every criticism of a moral philosophy I've ever seen boils down to finding an example where applying some high-level rule leads to conclusions that cannot be reconciled with the gut.
Also, the role of intuition in morality (and knowledge in general) is contested. I think that to commit to a system in which we only formalize our intuitions falls victim to three kinds of objection: those who have a reasonable case that we haven't caputered the intuitions correctly, those who doubt the role of intuition in moral decision making (several popular theories, from Kant's to Bentham's result in non-intuitive results in which their proponents will bite the bullet), and those who argue either that moral facts cannot be truthy, or that they can be truthy (but we always get them wrong), or they can be truthy (but we have no way of knowing which are true and which are false).
He could pick a form of utility based on ordinal utility that doesn't allow for interpersonal utility comparisons. In such a theory fewer things could be proven, but it would be easier (although still quite challenging) to prove whether an AI following that theory would decide to kill all humans.
> draining natural resources dry for generations to come?
That would count as negative production in my model as I define output as (global production-global consumption)/log(global production+global consumption), aka diminished excess product, not as simple global production.
> How does this treat personal autonomy
It explicitly does not, as there is an inherent trade-off between personal pleasure and freedom, versus overall conformity. But you can tune the function by adding overall comfort (total objective satisfaction) as a third parameter, which will unfortunately reduce societal comformity. My model is both non-authoritarian and non-liberalistic. It relies on modern day high speed communication facilities to exchange the input parameters for the utility function instead.
"A system that governs the behavior of a conscious agent that experiences the qualia of pain, within the same causal domain as other such agents, to arrive at the optimal dynamic equilibrium between minimizing individual pain and average group pain."
It's how I would formulate Kant's Categorical Imperative.
From that you get to concepts like consistency, conscientiousness, impartiality, and eventually the stricter form of the golden rule (which encompasses both "do unto others what you'd have done to you" and "don't do unto others what you wouldn't have done to you").
There's a cool looking symbolic logic version of it, too. My wife and I put it on our wedding poster. :)
Anyway, to echo other commenters here - it's not so hard to come up with ethical axioms (opinions/values) that are widely shared. The challenge is in keeping the reasoning consistent after that point.
So people end up doing the same thing, they just have more "logical" explanations of why their behavior is justified.
You could start with this list [1] of resources. I'd like to add Practical Ethics by Peter Singer.
https://www.reddit.com/r/philosophy/wiki/readinglist#wiki_mo...
You'd be able to say "The axioms are however not arbitrary" when (if!) philosophers have stopped arguing and the field of philosophy can say "for ethical axiom sets A-E, while having lots of earlier philosophical discussion, there's now a strong consensus in our field that they are wrong and flawed because of reasons, for any practical applications please ignore them and use axiom set F as that is the better one that addresses the known flaws in others". Currently, there's lot of philosophical discussion on ethics, but little of solid decisions and conclusions from these discussions; all the discussion is still at the stage of increasing ambiguity and introducing hypotheses instead of increasing our knowledge about reality by eliminating and discarding most of these hypotheses.
I think greater use of formal methods could really help with this. Imagine if there was e.g. a Coq library containing formalisations of all the common systems of axioms, and the various conclusions that followed from them. It would make exploration so much easier.
Hardly. There are three main branches and they all work on the same evidence and try to conserve most common sense conclusions.
> allowing you to get to pretty much any arbitrary conclusions
Absolute bullshit. You're either entirely unfamiar or lying.
Most importantly, if you post like this, you forego the chance to explain a more correct view to the rest of us. We all miss out that way. Even worse, commenting like this discredits the truth that it's trying to express (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...).
Of course it's extremely frustrating when someone is wrong on the internet, but it's all our job here to metabolize that frustration into something that doesn't make the situation worse.