Hacker News Reader

Comment by chasing | Hacker News Reader

There are actual living things we have difficulty proving are "conscious" and you get into really tricky territory trying to establish what might be "conscious" (or even "alive") in the world even without bringing AI into the mix. Even the people around us we can't prove to be conscious except in that we are also human and assume they have a similar first-person (subjective) view of the world and aren't just a biological robot running equations. Yes, you could literally be the only conscious being in the universe and the universe would indistinguishable from one with many consciousnesses.

View on HN

Edit:Shoot, I meant to reply to another comment. Leaving this here and linking it

---

IIT is the most coherent definition of consciousness I'm aware of https://en.m.wikipedia.org/wiki/Integrated_information_theor...

People dismiss it often, and it has valid academic criticisms, but most "lay" critiques I've seen seem to dismiss it because it gives things they don't consider to be "conscious like them" consciousness.

I think in general, we overexagerate qualities that humans have as uniquely special and important (intelligence, sentience, consciousness), even if the definition of them is fuzzy and any non-fuzzy definition is "too inclusive". I wonder if this is because we are collectively Identity building and creating an "other" out of nature, because otherwise we'd

1. Be much less special as we want to fee

2. Would have to consider a lot of inconvenient externalities for our reasoning (moral and otherwise) to be consistent

And both are slowdowns when you are bootstrapping a post scarcity society out of scarcity, so it'd culturally valuable to reify special qualities that we just determine "we" possess and "they" don't - because it's easier to unify on and allows more actions than making sure everyone can deal with the unfiltered reality of human... unremarkableness in an uncaring world, another social species like so many other with a temporary oligarchy on the planet (without wanting to drone to philosophically, I do think this aspect of Weltschmerz is underappreciated, especially seeing the anxiety amongst my peer group when the topic comes up)

> People dismiss it often, and it has valid academic criticisms, but most "lay" critiques I've seen seem to dismiss it because it gives things they don't consider to be "conscious like them" consciousness.

If your attempt at a formalism provides useful, actionable implications, and only really disagrees with people's intuitions on cases that are generally believed to be edge cases, or maybe that you also have good reason to believe are edge cases on different ground, then you might have a decent definition.

If your attempt at a formalism provides useful implications, but disagrees with everyone's intuitions on what the words you're using mean, you've probably defined the wrong thing. That's not to say that you aren't defining something sensible (you might still be), but strongly suggests it's not what you want it to be.

Selling IIT as a description of consciousness is roughly the same flavor of hubris as defining berries to include oranges and cucumbers and exclude raspberries and raspberries.

> Selling IIT as a description of consciousness is roughly the same flavor of hubris as defining berries to include oranges and cucumbers and exclude raspberries and raspberries.

Not trying to defend IIT, but the only reason your example works here is because from our human perspective (eating these plants) we find it absurd to include cucumbers and exclude raspberries.

So talking about hubris: Wouldn't plants (if they could) very likely have a totally different perspective? To then it would probably seem totally absurd that we use taste or shape of the fruit as a grouping feature and not something that might be totally more defining that plants can experience.

Of course it is intuitively understandable to look at ourselves and animals we understand when defining consciousness. Even like that it is hard to define. Trying to incorporate things that show conscious-like behaviors, but in totally human-unrelatable ways is not a bad next step, but it is hard to know where to draw the border. A rock could have conscious processes running in it on geological timescales — how would one falsify that?

You have acknowledged that there are valid academic criticisms of IIT, and for those wanting to get an idea of what those might be, a good place to start is with Scott Aaronson's responses in his blog: https://scottaaronson.blog/?p=1799

Note that the issue that you are concerned with here and the usefulness of IIT are separate concerns, and critics like Aaronson are not taking that position on the basis of the attitudes you claim are behind many 'lay' critiques.

I appreciate Aaronsons critique, he himself makes my main point of disagreements, i.e. that when constructing a physical systems that has the properties he describes (modified Vandermonde Matrix) it might give nontrivial surprises.

But even accepting that IIT is wrong in the "consciousness might imply high IIT score, but not vice versa" sense, I'm simply not aware of any definition which is as *attackble* because of it's formal specification.

This is not me going "well you don't have anything better", but me asking to be pointed to similarly strictly defined notions of consciousness

I found the conversation between you two interesting (as well as Scott Aaronson's post), so I figured I could contribute with a potentially relevant piece of reading: "Hard Criteria for Empirical Theories of Consciousness" (24-page PDF) at https://digitalcommons.chapman.edu/cgi/viewcontent.cgi?artic...

In the same ballpark as IIT, I've seen many mentions of GWT/GNT (global neuronal workspace theory). But the PDF covers that and many others. I don't have any particular "conclusion" myself, but it's just a fascinating discussion to be swimming in even when utterly clueless.

EDIT: And I forgot the most obvious link that I actually had read in the past, Stanford's page on conciousness talks about theories: https://plato.stanford.edu/entries/consciousness/#SpeTheCon

I am not aware of anything similarly strictly defined, but that is not, of course, evidence for the correctness or even usefulness of IIT, and nor does it somehow counter the specific objections that Aaronson raises.

IIT is a nice intuition pump, but it's missing essential parts. Consciousness should be related to survival, replication of information and its own energy costs. It's not just some integrated information for no purpose at all. The purpose is survival if it is conscious.

If you add that to IIT, then you suddenly get a new filter for what could be conscious - something that competes for information replication, not just something with high IIT score.

>> Consciousness should be related to survival

Is that not just your intuition too? Is it not possible to imagine an artificial consciousness that does not care one way or the other for its own survival?

No because survival is closely tied to evolution. By mere survival an agent creates its own structure and values. It can't evolve if it doesn't risk its skin in the game. The price of our present day intelligence has been very high.

But you can piggyback AI training on human culture and embody the agent in our environment so it can still function like a conscious biological agent. The AI is just passing the buck to humans, without us it wouldn't even have a goal.

On the other hand all living things have goals without human help. Maybe when AIs can self replicate their information into the future like us, without external help, they could be eventually conscious. We don't know yet how to make physical self replicators that evolve. Only nature solved this riddle so far.

BTW, AlphaGo was subjected to evolutionary techniques, and it shows. The agents had to duel with clones and past versions of themselves. Survival was conditioned on win rate. Suddenly AI can surpass humans in this little environment, but that came at the price of discarding so many unfit earlier versions.

It sounds like you've maybe read Steve Grand's book, Creation: Life and How To make It?

That is his entire pitch; if you've not read it I wholly recommend it.

> it has valid academic criticisms

That's probably because the rest of researchers ignore it like the plague. If everybody had to comment on it for some magical reason, it would be probably be burned down to the ground and then some.

IIT doesn't capture much or any of what we consider important about consciousness. If it's the most coherent definition we have, we don't have a coherent definition.

If other animals are just as special as us we can't boil them alive. Just imagine the inconvenience of delivering a less painful death to lobsters.

Also worth mentioning is the Mind-body problem i.e. Cartesian dualism and related concepts

https://en.m.wikipedia.org/wiki/Mind–body_dualism

Also interesting thought experiment is the philosophical zombie, lots to read by David Chalmers, etc.

> Yes, you could literally be the only conscious being in the universe and the universe would indistinguishable from one with many consciousnesses.

No, you think that's possible but it's naive. Your existence depends on environment and self replication of information. Your structure depends on it, your cognition. Are you saying you can be separate from the tree, and everything just a fake around you?

I’m saying there’s nothing you can measure to say anything — even another human — has the same sort of subjective experience (consciousness, a soul, whatever) that you’re experiencing right now, as you read this comment from inside your own head. You can make a guess, but it’s impossible to truly know. (This is a bit different from solipsism, if that’s what’s tripping you up. Although that, too, is a bit of a problem…) So any attempt to “prove” an AI is conscious is either going to fall into the hole of “well, how do you prove anything is conscious?” or is going to need a heavily distorted definition of “conscious” (like “seems kinda human” or “makes uncanny decisions we can’t fully explain”) to be true.

I think we can say that AI is conscious if we can observe its stack-trace showing it creating symbolic representations of its own processing and then processing those and using the same language for that representation as it uses for communications with humans.

In other words if we can observe a machine thinking about its own (symbolic) thinking, then I think it's conscious.

Consciousness is the ability to observe one's own mental processes and make sense of them symbolically. That is one possible definition of it.

You are right that we can't really prove that humans are conscious, because we cannot observe their thinking.

But, to prove that a machine is conscious all we need is to agree on a definition of what it means to be conscious, because machines can be easily inspected. And then find a machine that demonstrates features that fit the definition.

If we can't agree on what it means for something to be conscious then the whole question of whether some AI is or could be conscious is of course meaningless.

Consciousness is such a loaded word, and means something different depending on who you are talking to. For instance surgeons & anesthesiologists have a very peculiar definition of what it means to be conscious(ie conscious sedation); regular people might not agree it meets the minimum criterion of being awake and aware of yourself. I think a more important goal if we are designing artificial life might be whether or not an AI is conscientious(like morally).

> I think a more important goal if we are designing artificial life might be whether or not an AI is conscientious(like morally).

I have the feeling that attempts to create conscientious AI will just result in paperclip maximizers that have to use rationalization as an indirection layer to engage in their paperclip maximizing behavior.

Not much of an improvement, but realistically, what other result would be tolerated by whoever is building a DWIM AI to run a paperclip factory and for whom conscientiousness would just be a regulatory box to tick?

Good point, conscious and conscientious, is what we need

This might assume that only things that are measurable could be true.

If something only can "think" if it feed data it can not reflect about itself out of it's own volition.

This especially,

I'd be very skeptical of claims that, say, some complex ongoing control system, like a self-driving car, could be "conscious". But there's an argument someone could make.

But an artifact that has no data storage seems to fail any reasonable definition immediately. Maybe it could be part of something else that you can claim is conscious if you add storage, output controls, aims or whatever. But by itself the claim just seems preposterous.

Of course it can. Your laptop “knows” all sorts of stuff about itself and has all sorts of self-reflective processes that trigger without your specific input. Mine gives me warnings when my hard drive is almost full without me having to ask. This does not make it conscious.

But the GP is talking about specific properties of large language models like GPT-3. A laptop is indeed a self-maintaining, self-monitoring system, at a low level. A large language model isn't that. It is just a map from input to output and doesn't keep track of it's previous output or input.

GPT-3 actually does track the previous 2048 tokens of input and output for context.

Your laptop does not know anything, any more than your thermostat knows when it's hot or a toaster knows when your toast is ready.

Furthermore, one can argue that consciousness is a requirement for knowledge.

You’re wading into epistemology, a topic that’s been discussed and debated for millennia. Which maybe relates to my larger point that the concept of “consciousness” quickly gets out of hand to the point where it seems to me a little silly to say neural nets are “slightly conscious.”

(in an ironic demonstration of what I mean) I'm not sure I understand what you're saying - but is it that you can only call a process 'thinking' if it's producing some output that's not derivable from its input, nor hard-coded in the definition of the function, as it were?

Perhaps that's way off, I was just starting to think along those lines as I came to your comment, and it seemed it might fit, that we might be thinking along the same lines.

Just as dried up Tardigrades are not alive (reintroduced into an environment in which they can survive) Artificial Intelligence which runs decoupled from any notion of time and whos weights are frozen or changed without it being able to influence or understand the outcomes will not be able to derive a theory of mind about itself. Even if it did since it does contributes little to it's own objective regularization will remove these features. As ANN are trained today most architectures are dried up Tardigrades which get watered, dried&reset into the previous configuration during inference.

Right? Consciousness involves agency, self reflection, motivations, a sense of urgency. And a good deal of "theory of mind" and self denial.

I'll give it to cats and dogs, birds, the apes, and many other critters, but not AI. Those things are separate developmental tracks, and not included in AI research, from what I can see. They aren't going to spontaneously arise, and it seems to me that they all came before we achieved sentience at all.

Wouldn't that mean all humans cannot reflect about themselves? Even our heartbeat is a feed of data. To be especially smartassed the environmental conditions of the brain are essentially an indirect form, as temperature alone affects reaction rates. In absense of data and trying any random value a reproduction would fail in the same way trying to make an ice sculpture with water at 25°C would.

That's the problem. The neural net can't reflect on itself because it doesn't have a body and an environment to act in. It should be able to experience consequences of previous actions to close the loop on itself.

Reply on news.ycombinator.com