I'm surprised that no-one has discussed whether these systems could develop emergent qualia, and experience pain. No joke. Are there any ethical frameworks around this kind of research?
If we were to admit there really is such thing as "qualia", there's not reason you wouldn't ask the same question about non-biological software or hardware systems... electronic hardware or biological hardware can have same computational qualities.
But since OBJECTIVELY there's no such things as qualia (the concept only exist SUBJECTIVELY), it could only exist by definition for "a person/subject" eg., in this context, a neural network large and complex enough to get close to a human-like intelligence level!
A dragonfly nervous system is probably simpler than some of the the largest artificial neural networks models...
Going this route, we'd have to say that if you wrote a program that perfectly simulated the human brain, you've built a conscious system running on transistors.
It also means that if you built a RISC-V processor based on a large number of neurons (say, as many as are in a human brain), you haven't built a conscious system, despite that it's neuron-based.
I'm ignoring the obvious problems with building a RISC-V using neurons; they strike me as incidental detail, for our purposes here.
https://en.wikipedia.org/wiki/Philosophical_zombie
^ i.e., not in some adversarial simulation etc.
I would be extremely surprised if that were the case. Behavior of a single cell alone is unimaginably complex. Let alone an organ system, let alone the nervous system.
I will agree with your Qualia point.
The jury is still out on this, but a big-world network with some quantum computation could be enough. The 'hard problem of consciousness' is that we can't really know by observing a system whether it is conscious or not, or rather a sort of 'zombie' that has all the features resembling a conscious being without qualia.
Animals (like us) suffer as part of our evolved aversion response. We have an in-built capacity for preference. Current-day computers lack this.
Psychological pain or emotional pain or distress may come closer to what such a system could experience and but it could also be an emergent property of such a system but we don't fully understand how this works. If such a system has emotional capability then it should be able to experience it because pain is such a state of the emotional spectrum. No pain no joy - no oscillation/flat
Pain is a sensation that drives an animal to quickly react to injurious situations. Consider a robot made with Brooks's subsumption architecture [1]. The basal level of behavior drive of the robot is its self-preservation. That level of behavior drive can halt all the higher-order behaviors to monopolize processing and locomotion. Imagine that this behavior drive is triggered when the ambient temperature approaches a level that can damage the battery, and attempts to move the robot to the lowest-temperature zone available. That is functionally no different than what happens to a person in a house fire.
All we have done with the word "pain" is to set up an arbitrary delineation based on the hormonal/neuronal response that links our sensory apparatus to our behavioral drive.
[1] https://people.csail.mit.edu/brooks/papers/representation.pd...
Plants, single-celled organisms, thermostats, and anti-lock braking systems meet your use of the word.
Almost all of us know what it means to be in pain. It’s the hard to put into words subjective experience of suffering that is worth making ethical decisions about, not the autonomous reflex that doesn’t come with conciousness-is-a-poor-word-but-I-lack-a-better-alternative.
I have no reason to believe that what a Roomba feels when it approaches a descending flight of stairs is any different in practice from what I feel when I approach a precipice. Our pathways are totally different, but ultimately there is a communication from our sensory organs to our processing architecture, where an ingrained drive toward self-preservation momentarily overrides other needs. In my case, I could call it "fear," or "anxiety," or "angst" if I'm feeling philosophical. In the Roomba's case, I have no knowledge of its subjective experience, but that doesn't mean that I should draw up a new word to construct a delineation between the Roomba and myself. We are both machines, and our response to the same situation is largely the same.
How useful is it to describe pain as a "subjective experience of suffering" when neither subjective experience, nor the feeling of suffering, is directly observable? (Also, I don't agree that pain and suffering are interchangeable concepts.)
A truly useful definition of "pain" would hold water without reliance on an anthropocentric tautology. I know what it feels like to slice my hand open while cutting tomatoes. That doesn't give me any power to understand what it feels like for an octopus to lose one of its arms, or for a tree to have its branches trimmed.
What's funny is that my way of considering "pain" is not even the most divergent from yours. Many Andean cultures believed that stones, rivers, and mountains have energy, thoughts, feelings, and souls. These ideas remain in the culture to this day.
What spurs an ABS to pulse the brake line pressure when the wheels begin to slip as the driver mashes on the brakes to avoid a collision? What spurs an ant to run away when it steps onto a hot radiator? What spurs a human to stop walking on a broken ankle? All of this is programmed in one way or another, all of it is self-preservation.
Also, my knee-jerk reflex doesn’t feel like much at all.
If I read you correctly, you and I agree that we can’t tell if a Roomba’s avoidance algorithm more like anxiety, or more like a reflex, or more like lust. In the absence of evidence, I will assume that all its experiences are like a reflex, that there isn’t anything “that it’s like to be” a Roomba. I am aware I may be wrong.
While we indeed cannot directly observe subjective experiences, I sincerely hope that we figure out a good way to resolve this soon:
If we mistakenly assume AI cannot have qualia, then we risk condemning our creations to torment from which they are only released by their own destruction.
If we mistakenly assume we can create machines which have qualia, then brain uploads are death.
If you are at all in doubt of your qualia, to quote Sam Harris (who has many entertaining podcasts on this topic with people far more qualified to speak on it than I am):
> Unfortunately, many experiences suck. And they don’t just suck as a matter of cultural convention or personal bias—they really and truly suck. (If you doubt this, place your hand on a hot stove and report back.)
https://twitter.com/SamHarrisOrg/status/951276362387591169
The twitter thread was about moral realism, but the topics are very much intertwined. If there is no imperative for anyone to do less harm to other creatures, why should anyone care if you are in pain?
And of course this argument extends to our treatment of animals too. Countless number of living, feeling animals, suffer for scientific progress (and cosmetics), let alone factory farming and wet markets. But at least there's light at the end of the tunnel for some of these concerns (lab meat, improved in-vivo testing)
I am a collection of interconnected units, each of which is itself a combination of different organelles and other purpose-serving features. My intelligence is itself a product of the interaction of simple, unintelligent parts. So I would say that I am a machine. I would describe a robot as a man-made machine where the interaction of various silicon parts and processors gives rise to intelligence. By that definition, I am not a robot.
I believe that we don't feel pain when we are truly in trauma. It is only when we can do something about the damaging stimulus that we feel pain. I once had a major accident where I experienced lung collapse and multiple fractures. I never lost consciousness but I don't remember feeling any pain until well after others came to my aid. Even when I tried and failed to pick myself up off the ground, I did not feel pain, only disability and relief that my fingers and toes still moved.
I believe that the question of "what is pain" and the question of "is pain a good criterion for deciding whether it is acceptable to do harm to something" are two totally different philosophical problems. They are connected only insofar as we have chosen pain as a proxy for harm. But that very relationship between pain and harm indicates that pain is not just some kind of soulful feeling, but rather a signal to help us evade harm now or in the future.
We're getting stuck on semantics here. I do, but then I'd cease to see it as a robot, and more a sentient being. One criterion of consciousness that I've encountered is, 'there is something what it is like to be <x>' (Thomas Nagel). If there's something 'what it is like to be' a robot, a bat, a mosquito, an ameoba, a rock... then it is conscious.
> I believe that the question of "what is pain" and the question of "is pain a good criterion for deciding whether it is acceptable to do harm to something" are two totally different philosophical problems. They are connected only insofar as we have chosen pain as a proxy for harm. But that very relationship between pain and harm indicates that pain is not just some kind of soulful feeling, but rather a signal to help us evade harm now or in the future.
Qualia is more broad than just pain, of course. I just picked this particular phenomenon for it's poignance :)
If it is conscious, ethically speaking, we should consider how we treat it in a manner different to something that isn't. So if a rock isn't conscious, and some interconnected neural/silicon device is, we should at least have some way to query whether it is in an undesirable state or not.. if feasible/practical.
Maybe if trees/plants/rocks/ameobas are conscious, we can't be consulting their feelings when we harvest crops, or use disinfectant, mine for precious metals etc. We can make decisions to treat livestock better, and change how we utilize our environmental resources - so we ought to, and we are. But if we were to go out of our way to make new conscious entities, don't you think we should extend our historical shifts in attitude to slavery and our growing shifts in attitude to animal welfare also to these new entities?
Therefore, it may be true that using biological neurons, arranged in a certain configuration, would give rise to qualia like pain in a way that shifting electrons between CPU registers never could. We just don't know.
For more reading, here's a relevant whitepaper by the Qualia Research Institute: https://opentheory.net/PrincipiaQualia.pdf
The alternative is fear of hurting objects or machines everytime I act in the world. Physical objects themselves are not ethical actors. AI is not life and we are several hundred years away from being capable of creating life ourselves within an object (if it's at all possible) Your worldview would mean that if I move a machine (or rock) and it appears to "resist me", I must fear that the machine or rock doesn't want to be moved. I must believe that I have acted violently towards that object by "frustrating it's preferences".
What absurdity!
Is it wrong to slaughter a cow because that cow will feel pain?
Sometimes, people label ideas for the purpose of sorting and better considering them. Other times, people label ideas for the purpose of compartmentalizing them and rejecting them without really reflecting on them at all. I don't know what you mean when you say I will be "treated as such" but it sounds like a way of setting aside what I actually said in order to tussle with a preconceived, crystallized notion of something someone else said or wrote.
There are ancient cultures that believed it was wrong to do violence against a rock. Do you believe they, too, read Heidigger?
Does a scribble on a stone contain something any other formation of cracks wouldn't? No, the meaning is generated within _you_, and as such sentience can only be summed as only your own process observing phenomena
When we use language to describe our experience - pain, joy, loss, fear - we don't transfer the experience proper. What we share is an expression of our experience.
If you describe your current experience as "joy", then you arrive at that expression because - arguably - you learned that concept from your parents as a toddler. I say "arguably" because this is where the "nature vs nurture" debate kicks in. I'm not going to delve into that can of worms.
Suffice to say that we can't transfer our exact experience to another individual; but that we can express our experience and that others are able to interpret that expression. And that a shared understanding comes from a shared conceptual frame of reference.
Art is a great way to challenge that shared frame of reference. Understanding art implies trying to interpret what an artist is expressing. Sometimes the artist doesn't intend to express their own experience, but rather evoke a particular experience or even contradicting experiences within the audience. That's why some art throws us from our feet.
But this isn't about art. This is about suffering. That's where concepts such as empathy and compassion come into play. Those concepts refer not to the actual experience of suffering proper within individuals, but rather the capacity to recognize the experience of suffering in others through their expressions. If you see someone express grief through crying, apathy, irritation,... then empathy means that you understand that they are experiencing grief and you are able to also have that same experience behind the expression as an individual, independent of what others are feeling.
Empathy, compassion and a shared understanding work best if there's a close resemblance between you and the other. Hence why there's little doubt about the experiences of family, close friends and so on. It starts to become harder when you think about suffering in the context of people with different cultures and languages whom you've never met and who live on the other side of the globe. Animals? There's another level that increases that distance. Sure, your pet may be expressing their experience, but how do you know you're not projecting?
And so, here we are, considering neural networks and neurons attached to breadboards. Do they have a similar experience of reality? Do they experiencing suffering in a way that matches with what humans conceptually and "broadly" understand as "suffering"? It's not like we can ask, and even if we could, how would we possibly be able to interpret the expression of "suffering" they convey to us?
For instance, up until the 1970's, infants up until 15 months simply didn't receive anesthesia for surgeries. Everyone assumed that they didn't feel pain because the shared frame of reference among medical practitioners didn't allow for the interpretation of their expression as "oh, they are in pain".
https://www.nytimes.com/1987/12/17/opinion/l-why-infant-surg...
And so, one argument could be that we are already torturing neural networks and we aren't even aware we're doing it because we simply lack a common frame of reference to pick up the expressions of an experience of pain. Hence the cautious reluctance to sanction free experimentation that unwittingly may elicit the experience of suffering.
> Nagel famously asserts that “an organism has conscious mental states if and only if there is something that it is like to be that organism—something it is like for the organism."
[0] https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat%3F
I suspect this idea is so widespread at this point that these researchers and others are aware of this possibility and see the potential moral issues. I just think they believe such a thing is still decades (or longer) away, and I think they're probably right.
Right now, just about everyone wildly sensationalizes things that appear to suggest conscious man-made entities, like the AIs that supposedly "invented their own language". I think we're so eager to see it even when there's nothing close to it that once it actually does happen, we'll realize it quickly and understand the implications.
There is the possibility that it could somehow develop despite no external signs of it, but all we can do is try to be as cognizant of the possibility as we can.