Hacker News Reader

Comment by Veedrac | Hacker News Reader

So this article does not actually defend its claim, it just gets mad that some AI researcher expressed their opinion, makes an unsourced (albeit probably correct) claim that the criticized opinion is a minority position, and wishes really strongly that people would be less excited about this thing that they are not as excited about.

Meanwhile in academic philosophy, it's totally OK to conjecture that, say, subatomic particles are ‘slightly conscious’ and nobody tries to tell them that they are not allowed to have an opinion.

Here's a hint, if you want to refute an idea, refute the actual idea, don't just tell the person in so many words that they don't have the social status to say it. Yes this post wound me up a bit, how could you tell?

View on HN

You're asking for a refutation, and ordinarily that's a good thing to aim for, but in this case was the original claim clear enough to be refuted? I think we don't even have a good definition for consciousness and we certainly don't have agreement over what would constitute evidence for it from the view of an outside observer, and the original claim doesn't attempt to provide any evidence, and so doesn't even imply an epistemic position. How can one refute something which is so vague?

If one considers oneself entirely unsure about what properties consciousness has, maybe one shouldn't be trying to refute twitter posts that hypothesis certain things might be conscious. Maybe one should do the more principled Bayesian thing and just allow oneself to be uncertain proportional to the amount you don't know.

If the article stuck to “we don't have any good reasons to believe that neural nets are conscious,” I wouldn't have had this objection. Either you get to say you know some properties about consciousness that allow you to coherently argue that some things are or are not, or you don't claim that and then you don't get to make that argument. The article wants to have its cake and eat it too, making a strong stance about what their model says while also claiming to neither have nor hope to have a model.

I do actually think there are useful things you can say about what is and isn't conscious, but my comments here aren't really about that, they are about this annoying argument style where you attack things from whether they conform to social norms, rather than the merits of the position.

It’s up to the person proposing a new term like “slightly conscious” to define what that means. Without that I can say no neural nets aren’t blubbery muffins and it’s just as valid as any other argument.

Quite simply I don’t agree or disagree because the guy failed to clearly communicate.

Agreed. The backlash to the OP's article reeks of religion. A few AI zealots pushing the burden of proof upon their naysayers under the pretense of science. Meanwhile, actual science is far closer to basic concepts like:

https://en.wikipedia.org/wiki/Hitchens%27s_razor

https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy)

This is nonsense. The statement isn't meaningless. It's a tweet, not a formal conjecture.

Sure, no one knows what exactly consciousness is... or what slightly conscious is. The idea of protoconsiousness isn't that much of a stretch. If a biologist had suggested some plants have protoconsiousness, no one would have whined.

So what if someone working on NLP has an opinion about consciousness. Why is their opinion less valid than someone else's. It's not less informed.

I didn’t say meaningless, I said he didn’t communicate clearly. He can always add more tweets on the subject to remove ambiguity.

Suppose you walked past a billboard that just said “Jump.” That’s grammatically correct and would be meaningful with context, but sitting alone you simply have no idea what idea was supposed to be communicated. Perhaps you missed a different sign, perhaps it’s art or an in joke, but sitting alone it’s simply ambiguous.

> Maybe one should do the more principled Bayesian thing and just allow oneself to be uncertain proportional to the amount you don't know.

I think this still fails in the sense that the distribution representing one's beliefs is only defined with respect to a set of hypotheses. By invoking Bayes, presumably you acknowledge that different parties will come to the conversation with different priors. But critically, one can only update one's beliefs given both evidence and a likelihood function related that evidence back to hypotheses in the domain of one's beliefs.

The "our take" summary near the end of the article explicitly says that they would regard the claim as reasonable given the right explicitly definition of consciousness.

> The reality is that we don’t have a widely accepted definition, let alone understanding, of consciousness. Claiming that we have already replicated such a nebulous concept with computers seems improbable at best. Our take is that the claim could also be reasonable, but only if a particular definition of consciousness was specified as well.

Many orthodox people speak as though it were the business of sceptics to disprove received dogmas rather than of dogmatists to prove them. This is, of course, a mistake. If I were to suggest that between the Earth and Mars there is a china teapot revolving about the sun in an elliptical orbit, nobody would be able to disprove my assertion provided I were careful to add that the teapot is too small to be revealed even by our most powerful telescopes. But if I were to go on to say that, since my assertion cannot be disproved, it is intolerable presumption on the part of human reason to doubt it, I should rightly be thought to be talking nonsense. If, however, the existence of such a teapot were affirmed in ancient books, taught as the sacred truth every Sunday, and instilled into the minds of children at school, hesitation to believe in its existence would become a mark of eccentricity and entitle the doubter to the attentions of the psychiatrist in an enlightened age or of the Inquisitor in an earlier time.

Here's an opinion that virtually nobody holds. You want me to adopt that opinion? You'd better have some reason for me to do so - some evidence.

Here's an opinion that 99% of people hold (or 99% of those within a field - same effect). You want to persuade someone to not hold it? Better have evidence.

It's not exactly that the burden is on the one making the claim, or on the one denying the claim. The burden is on the one who is trying to change minds from their existing beliefs. That one needs something convincing - principles, reason, or evidence, with evidence being the best.

"A sense of awareness." Does that work as a definition of consciousness? Also "sensible inwardly," and where we might get into trouble: "knowing."

Do you think AI has a sense of awareness?

I can't see one reason to ascribe a sense of awareness to any current form of AI, as world-changing and amazing as it is already. Even the I in AI tugs at me because I feel it might be a metaphor taken literally. As in, McDonalds is a hamburger empire but surely not a real empire. AI "knows" something in a way similar to how the dictionary "knows" the definition of a word. Beware of taking metaphors literally; there be dragons (but not real dragons)! AI has advanced beyond symbolic processing, it processes information in fantastic new ways, but it strictly speaking understands nothing. We take the processed information and we understand it (or don't). But maybe there's a case to be made that I'm being unfair, and it's actually justifiable to say that AI "thinks" without being conscious, aware, or sentient.

On panpsychists, if their heady intuition is somehow not deluded and a form of awareness or consciousness is a fundamental quality of matter or in some other way ubiquitous, maybe like "The Force" in Star Wars, that's clearly not the way that people would mean conscious here. An appropriate way to discuss AI in that context would be, "AI is slightly conscious just like everything else." And if consciousness were somehow a quality of all matter it still wouldn't mean that, say, a chair is conscious as a chair or a tokamak as a tokamak, or even a carrot as a carrot, but rather that the matter that makes up those things and everything else has some kind of awareness.

In other words, the original author was not even wrong.

> Here's a hint, if you want to refute an idea, refute the actual idea,

Science doesn't work that way - https://en.wikipedia.org/wiki/Falsifiability

Definitions don't fall under falsifiability though.

If I define "zorg" as colour #ff45b3, arguing "this is not the case because it can't be disproven" is rather silly.

Similarly if I say AI is "slightly conscious" as per my definition of consciousness, and a critic says it's not given their definition, they're both right (but in a totally uninteresting manner).

Which science is that, that requires falsifiability?

English? A made up word that gets used certainly is a worthy object of study in that science. Similar points can be made about all humanist sciences.

Natural sciences? Say, Biology. Fictional, and therefore unfalsifiable, biological systems, in the context of the search for extraterrestrial life or just generally (e.g. can we have living silicon-based life?) are objects of study.

Physics? Only experimental physics requires falsifiability. That a large part of it is in fact searching for falsifiability illustrates, I hope, that falsifiability is not a requirement for 99% of Physics. String theory is famous for being unfalsifiable.

Math? Unfalsifiable theories are seen as a good thing in math, and are certainly objects of study. That we don't have a good unfalsifiable set theory for over a century is one of the great problems in math.

You might say I'm a utilitarian. But so is everyone else, despite a few people being confused. Of course, that's exactly what a utilitarian would say.

String theory is falsifiable, just it requires so much energy that we cannot operate with in any foreseeable future.

> String theory is famous for being unfalsifiable.

Yeah, but this is a reason why lots of physicists disregard string theory as a dead end.

The context here is that, when AI was overhyped last time, the whole field collapsed to the level below ground.

You see, overhyping doesn’t exactly help the field:

(1) researchers need consistent investment over a long period of time.

(2) a large amount of blind money goes into fake projects and false promises.

(3) after the bubble pops, the “AI” will be bashed hard for being not very profitable.

(4) here comes another winter.

So, for academia, it’s better to get hyped slightly less and run longer, instead of go brrr and die tomorrow.

hmm, I am not sure this is a fair assessment, the portion on "Experts largely agree that current forms of AI are not conscious, in any sense of the word. " provides sources and a brief argument. Sure it's not a super long defense of the stance, but then again this is mostly an overview of what happened with all this twitter drama and not a full argument about this topic.

Also, it outright states "Granted, the claim could also be reasonable, if a particular definition of consciousness was specified as well."

> "Experts largely agree that current forms of AI are not conscious, in any sense of the word."

Experts in what? There _are_ no experts in consciousness, at least in the lay sense.

Looks like you are getting downvoted, but it's true. Defining exactly what it is to be "conscious" is a nearly impossible problem to solve, even having spent your life studying it. I'm not personally even convinced that "cogito, ergo sum" is even correct.

It's true that "conscious" may be difficult to define, but it's almost impossible to come up with a definition for which there aren't exisitng experts.

'cogito' is a stumbling block in itself really.

Every word in "cogito ergo sum" can be questioned.

Perhaps this is the point. If you don't have an agreed upon definition of the word, it is not a useful tool. A claim o consciousness, if that claim is meaningless, isn't useful.

But aside from that, there is a lot of philosophy on what consciousness is (https://en.wikipedia.org/wiki/Consciousness has some of it). And those people, especially philosophers in the crossover of computer systems/intelligence and general philosophy are "experts".

The article even mentions that the CEO, Sam Altman, says that it’s not conscious in any way that you would use the word “conscious”. It’s like porn that way. You know it when you see it. And a computer is not conscious in anyway you would recognise.

And that’s why science communication matters. Having some super excited researcher talking publicly without an editor or someone to ground them just gives wrong ideas. Imagine the number of enthusiasts who did not read any rebuttals and now think AI are becoming conscious.

Experts in AI.

The statement "Experts largely agree that current forms of AI are not conscious" connects to two pieces of expertise: expertise in consciousness and expertise in AI. It is plausible an expert in AI might have the background to state with confidence that AI is not "conscious" in any meaningful sense of the word.

Experts in AI may be experts at using AI techniques or designing AI systems, not necessarily in deciding in what is or isn't conscious.

What is or isn't conscious is arguably as much (or maybe even more) a political, moral/ethical, philosophical and religious question as it is a technical one.

It's sort of like the question of what is or isn't alive, or what is or isn't a person -- questions which we know are central to the abortion debate, euthanasia, medical experimentation, animal/human rights, etc...

Consciousness is just as much an ethical/philosphical/religious mine field, and I'm not sure I'm comfortable handing over its definition to some random people who just happen to be good at, say, designing neural nets.

>Consciousness is just as much an ethical/philosphical/religious mine field

Religion is the observation of chemical effects on the body, psychological warfare on the mind (ie fear of an unknown all powerful entity that created everything around you) and then some sort of semblance to create a social rules based order out of chaos, whilst also attempting to understand whats going on.

Philosophy is deep thinking, sometimes using abstract examples to conjure up ideas which might work on the social order to avoid chaos.

Ethics is another attempt to install a social order on individuals and groups which can include society as a whole.

Consciousness is an entity's feedback/sensing from reactions within its container (body/app), feedback/sensing from the environment its in, the ability to remember past experiences and the ability to interact with itself (feed or rewrite code) and the environment to further extend its own life, acquire more knowledge and perhaps reproduce if immortality is not an option, but if immortality is attained being able to reproduce becomes an ability that might not be carried out for other reasons, like the need to live within the constraints of its environment.

Every lifeform on Earth has evolved through trial and error to survive in an environment through chemical changes over millennia, a bit of Darwinism, pot luck and obviously remembering what works in order to continue its survival.

Reflexes could be considered unconsciousness which you may or may not experience (typically chemically based but what drives the nervous system other than some calcium/sodium/potassium acting as a low current voltage electrical source), for example, breathing most of the time its in the background, you are not conscious of it, but remembering tells you that you are doing it, but sometimes we become conscious of it due to other chemicals which might be raised affecting the body and brain.

So I'd say provided the AI can remember what works in order to continue its survival by avoiding what kills it if its had a close call, and it can seek out what helps it survive, like powerups in a computer game, or jumping from computing device to another where it can detect things like battery backup/UPS and reproduce to further aid its survival, it can explore its environment and perhaps adapt to new environments ie jump from a laptop/desktop to a network switch and handle the different instruction sets which exist with different cpu's, then I think you can say its conscious. Lifespans/timelines are going to vary, but if some code displays this within a computer, then I think you have a true AI even though we might not be able to communicate with it other than perhaps by scaring it off like the fleeting glimpse of something out in the wild, that hasnt been caught yet.

I'm torn over the reproducibility though because if it could survive indefinitely in its environment thats no different to cancer cell lines like https://en.wikipedia.org/wiki/Henrietta_Lacks

However if it could survive in its environment perhaps using reproducibility as a survival extension mechanism that would then mean having to pass on knowledge because otherwise you would have AI able to learn from its environment but its not passing on TLDR's from previous experience, likewise I think an AI would need to be able to move around in its environment like go from a network switch to a computer or even incorporate a network switch along with other parts of it running on a desktop/laptop.

And if it can communicate, could it communicate using Human Interface Devices, or other things which would enable it to explore its environment, just like we can send probes off into space or undersea.

Some computer viruses fit some of the definitions above but not all.

So yes, "slightly conscious" should be qualified better, its too general, but it might stimulate another round of funding for those wanting to explore the field.

For example, should the ability to traverse new environments be included or not? Not all lifeforms on this planet can do that, eg fish excepting those taken out of water for a shortwhile. But most mammals can survive for a period of time in and out of water, like dogs, bears, etc, etc. Its a tough one because different animals have evolved to survive in their environment and that environment could include abit of land and water, like penguins.

This is the problem until consciousness is defined, what is slightly conscious?

Edit: If I had to say is there a company/entity which is getting close to be classed as an AI, then of the publicly available information out there, Windows is currently closest to becoming an AI. Its got https://copilot.github.com/ which probably wouldnt take much to hook up and have it rewrite itself. It already runs on a wide array of machine architecture and thus various environments when considering ruggedized laptops, phones, servers, office & home spaces. Its update mechanism is both a central server farm and peer to peer which wouldnt take much to get it updating using other communication methods beside network access, in some respects it already does this when thinking about mobile data networks. I havent seen anything like this from Facebook, Google, Apple or anyone else yet, but that might be because its not public knowledge.

Do we even have a consensus on which animals are conscious?

It is possible to define an upper boundary for "this is not conscious" and a lower boundary for "this is conscious" with grey area in between them.

Thus, even if we cannot clearly state for any given animal whether it is or is not conscious, we can still clearly state that, say, a coffee maker is not conscious, even if it has rudimentary processing capability, or that a person is.

As I implied in another comment[0], I believe it would be both possible and valuable to construct a set of conditions that we collectively feel are necessary, if not sufficient, to define consciousness. That way, we could at least rule it out as long as no AI meets those minimum standards.

[0] https://news.ycombinator.com/item?id=30409569

> we can still clearly state that, say, a coffee maker is not conscious, even if it has rudimentary processing capability

a coffee maker might include a temperature sensor (thermistor) and a heating element. does it predict the future? yes: it has an expectation of the future temperature given the current state. does it model itself as an agent? to a limited degree: it considers the future effects to the thermistor which its immediate control of the heater has, it might even run hypotheticals over a set of different heater controls. throw in a pressure sensor or a flow meter and some valves for more fun.

now, i haven’t read much literature post 90’s on this, but one of the more prominent components of consciousness seemed to be: modeling yourself as an actor within a larger setting, and using that to produce outputs based on predictions and thereby cause a specific future(*). does the coffee maker qualify?

also i don’t think many people think of consciousness in a binary sense. that grey area you describe may be less of “we don’t know if these are conscious (binary)”, but rather “we don’t know if these things have experiences which are relatable enough to our own to be worth considering in the motivating questions (i.e. ethics).” and the things at the bottom (the coffee maker) are more like “whatever experience this thing has is so far removed from our own that we’ll never be able to understand it in human terms (like pain and suffering or wants and desires).”

(*) though some people like Dennet argue that the conscious experience might not always be involved in the causal part of this loop.

Do you believe it makes sense to even claim that other humans are conscious?

If conscious is defined as “to have subjective experiences”, then I don’t believe “other people are conscious” is coherent.

The argument I hear usually is that other bodies are constructed like my body and I’m conscious therefore they are probably conscious too.

But I think this completely misses the point. The issue is the proposition itself. How can that proposition be translated into empirical claims? If the answer is just that other bodies are like my body, then conscious is just a fancy synonym of “is a human being”.

>If conscious is defined as “to have subjective experiences”, then I don’t believe “other people are conscious” is coherent.

Coherent but about as likely as Last Thusdayism. You have to factor the probability and potential mechanisms of what might give rise to consciousness and if you do you should have a very low probability on other humans not having it, just as you have a low probability on a random human you meet physically not having a brain without having to confirm by looking inside their head.

Right, I have a low probability belief that a random human won’t have a brain in their head.

But in principle, I could open someone’s skull and look for a brain. How would I examine another human body for consciousness?

You can't but that doesn't change the mechanism of arriving at beliefs. You yourself can also never physically check say Plato's skull either but I doubt that deters you from thinking he had a brain.

> You can't but that doesn't change the mechanism of arriving at beliefs.

I’m trying to question the coherence of the proposition itself, and not trying to refute an argument in favor of it.

> You yourself can also never physically check say Plato's skull either but I doubt that deters you from thinking he had a brain.

The Plato’s brain question (PB) confuses me, but here is my attempt at an analysis.

First, I think PB is tricky in the same way that all historical propositions are tricky. And I think this trickiness is different than the trickiness around the proposition “other people are conscious”.

Second, here is a scenario that I think is like PB. Consider the proposition, “The Great Wall of China exists.” I’ve never been to China, but say you go and tell me that you saw the wall. And in principle I could go see the wall for myself. But now what if the wall is destroyed before I go to see it? Was the proposition that, “The Great Wall of China exists”, coherent before the wall’s demise, but now incoherent?

But if you tell me, “I am having subjective experiences”, how can/could I ever in principle examine this for myself, as I could the Great Wall?

>I’m trying to question the coherence of the proposition itself, and not trying to refute an argument in favor of it.

The point is that just because you can't prove something to yourself it doesn't make the belief incoherent. And you can use your other tools to arrive at likelihoods for said belief.

At any rate, there's plenty of things we believe about say quantum mechanics without being able to observe them directly by using theory and observing secondary effects.

In theory it's also possible to eventually have an exact definition of what we mean by consciousness, figure out which parts of the brain are relevant in what ways and observe secondary effects and use theory to confirm others have it same as with QM. It's all caused by physical effects and at a higher level than things we already reason about just fine.

> The point is that just because you can't prove something to yourself it doesn't make the belief incoherent.

If a proposition doesn’t make predictions about my empirical experience then what is it saying?

And if a proposition does make such empirical predictions, than it’s effectively just a name for those predictions.

That’s essentially the view I’m coming from.

> At any rate, there's plenty of things we believe about say quantum mechanics without being able to observe them directly by using theory and observing secondary effects.

I don’t know any physics so I can’t comment.

Why are propositions about consciousness different from propositions about faeries?

>Why are propositions about consciousness different from propositions about faeries?

Ultimately they are not. It's just that the evidence points towards an extremely low chance of fairies being real and extremely high chance of others being conscious in a comparable manner to how you are conscious.

So then is one of these propositions true?

Every North American faerie weighs more than 10kg.

Every North American faerie weighs 10kg or less.

Both of those propositions are true. I don't see the connection to consciousness though.

Are there truths that are neither tautological (mathematical) nor empirical? I would suggest those statements are only meaningful if in principle we could measure the weight of a faerie.

As another example, I suggest that “Every idea weighs more than 10kg” and “Every idea weighs 10kg or less” are both not meaningful, because it is not possible to measure the weight of a given idea.

And so consider a claim like, “Consciousness has property X”, how can I decide whether or not such a claim is true?

edit: That last claim should have been, “Some other person’s consciousness has property X”.

The difference is that claiming consciousness may kill the entire field of ML research when non-experts decide to wade in and start lobbying for regulation. You don't want misguided groups like PETA meddling with regulating neural network research. FAANG won't be affected much either way but your average university will.

When academic philosophers say that subatomic particles are 'slightly conscious' and we (normal, everyday people) commonly say that a person who is alseep is unconscious, then it's obvious that we're using two completely different meanings for the concept of 'consciousness'. Academics (scientists in particular) should stop making 'WHOAH!'-sounding statements to try to make it onto '/r/IFuckingLoveScience’. It exacerbates the trust problem.

Both statements appear un-falsifiable, which makes the entire discourse about as useful as debating religion.

many religious claims are falsifiable yet debate is still useless.

Preferences are unfalsifiable.

Not to oneself.

The burden of proof is on the person claiming something is true.

So if I claim it is true that neural nets are not conscious, the burden of proof is now on that claim?

The burden of proof is on the person making an assertion. The original claim was not an assertion, it was that they "may be slightly conscious". The article linked here is the only one that made an actual assertion, which is that the original claim was categorically false.

In short, I agree with you. The burden of proof is on this article to demonstrate that neural nets are not conscious.

The null hypothesis is that things aren't concious until there is evidence that they are.

And what evidence is that, specifically?

Can one argue that humans are even conscious? Doesn't consciousness require some form of free-will?

See, now we're running into all sorts of problems.

First: What is consciousness? That's the first thing we need to do. Come up with a definition that we can both agree on absent any particular example. So we can't hold up a human and just shrug and gesture in its direction.

Now. We need to prove that humans fit that bill. And in a way that precludes other philosophically possible scenarios. Or at least agree on some fundamental axioms. Like assuming that no one is a brain in a vat. But if we're trying to prove humans are conscious and not simply automatons, can we even dismiss solipsism? Because if humans are not conscious, then I could be a brain in a vat and they're just automatons running for a separate reason.

And the truth is, we can't get past even those two hurdles. We assume humans are conscious because we kind of define consciousness as just shrugging and gesturing in its general direction.

Even if we call consciousness "awareness of self", how do we prove something is aware of itself and not just making the claim without understanding.

The null hypothesis is not your prior!

https://en.wikipedia.org/wiki/Null_hypothesis

In Academic Philosophy people can say subatomic particles are slightly conscious and no one would critique them, including other Philosophers?

"it may be that today my fingers are slightly conscious"

Please refute.

Reply on news.ycombinator.com