A Google Employee Fell for the Eliza Effect | Hacker News Reader

A Google Employee Fell for the Eliza Effect

theatlantic.com

30 pointsandsoitis22 comments

I never thought about all of this in relation to animal cognition and I really like the point about animals, mostly because we have been recently learning that animals have far greater mental ability than we like to admit.

And I'm amused that at some point we would have considered someone actively an idiot for believing animals have complex stuff going on up there. Makes me wonder if that guy was tricked by a dog once or something and never really got over it!

> Following Alan Turing’s 1950 paper, anything less than 70 percent accuracy by the judges would constitute the machines “passing,” so LaMDA would need to fool just nine of the 30 judges to pass the Turing test. If I had to, I’d bet (though not a lot) that LaMDA would, indeed, fool nine or more of the judges.

One thing I didn't know reading Turing's original paper is he predicted how big he thought a computer would have to be to pass his test.

> I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 10^9, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning. https://academic.oup.com/mind/article/LIX/236/433/986238

Turing said it'd take 10^9 bytes (words?). Google has a storage system called Colossus that has 10^18 bytes, so clearly they must be slacking off if they needed that much room to finally pass Turing's test.

Those early pioneers of computing were able to do so much with so little, that I think they couldn't possibly imagine how bloated things would get. I think they were assuming that developers would only get more efficient, rather than much much less. Certainly naive, but in my opinion understandable.

Bloat keeps the infinitely replicable labor economy afloat.

>I predict that within 10 years, computers will be twice as powerful, ten thousand times larger, and so expensive that only the 5 richest kings of Europe will own them.

Professor Jonathan Frink, 1960s

Interesting article. Purely reacting to the headline, I think a Google employee fell for the effect of wanting to be famous. Big tech companies are having to deal with the reality that in certain areas (integrity, privacy, AI) employees with sensitive info can become celebrities if they decide to leak, do a profile in WSJ/NYT/WaPo, and hit the speaking circuit.

Could humans convince an AI or an alien intelligence that we were conscious? What evidence do we have? Maybe consciousness feels different enough for different beings that we wouldn't even recognize the words we used for the experience once we learned each other's languages.

Maybe, or he is trolling

https://old.reddit.com/r/meme/comments/vga9q1/google_has_hir...

And a Google manager fired a guy raising an ethics concern.

had he even understood what he was talking about he would have known it wasn't a concern at all.

i venture to bet that he wasnt fired for raising the concerns, he was fired for raising them in an extremely poor way.

This is the important bit:

> I believe that AI systems can, in principle, be “conscious”/“sentient”/“self-aware”/moral agents/moral patients—if only because I have not seen any compelling arguments that they cannot. Those arguments would require an understanding of the nature of our own consciousness that we simply don’t have.

There are too many philosophers who are insanely over-confident that their wooly evidence-free idea of consciousness is the right one, simply because other philosophers have written a lot about it.

Nobody knows what consciousness is exactly so people proclaiming that this AI can't be conscious are wrong. I don't think it is conscious yet. But that's a different matter.

What does it mean that something "can't" be conscious? A cup of coffee could be conscious -- you can't prove that it doesn't have some kind of conscious experience despite its inability to act independently. But we observe consciousness in ourselves and apparent consciousness in other beings more or less resembling ourselves, and that gives a general idea of what kinds of things might possibly be conscious.

There is no indication that some numbers, stored in memory, and an algorithm for applying arithmetic to those numbers to assign probabilities to tokens might be conscious. If it were conscious it would be so significantly different from the kinds of things that seem to have consciousness that at the very least some additional evidence is warranted.

I'm as confident that lamda is not conscious as I am that my coffee isn't conscious, so I'll shorthand that by saying that neither of them "can" be conscious. Sue me!

Agreed. One distinction that I've found people have trouble in debates regarding lamda is separating the idea that any billion parameter ML algorithm could be conscious vs lamda in particular being conscious. I think the jury's out on the former (I'd bet you could create a reactive embodied agent with learned environmental goals in <1B parameters given the right setup, which is at least a stepping stone), but for the reasons you describe here, lamda could only be considered conscious by a definition so broad as to declare all digital computation as conscious, and the reasons for that come down to understanding the network architecture and prediction task.

It's also interesting how many people say, "it's merely acting like a sentient AI" as if an AI researcher at Google hasn't thought of that already. He is either trolling for press or truly believes that this is different than anything else he has encountered.

It's strange how a microprocessor is regarded as wholly dead and unconscious when running a spreadsheet or an operating system, but the moment it starts producing outputs in natural language, it somehow has a claim towards sentience due to the confusion caused.

I'm a junior philosopher here. I just did an undergraduate dissertation on Wittgenstein and problems for AGI. Much confusion in the field, especially from certain philosophers who couldn't even programme a microwave, nevermind comprehend what a processor is doing at the low level switching bits.

We also know absolutely loads about consciousness and sentience. Even a child can understand how we can recognise and properly predicate the word "conscious" to various entities both alive and dead. Philosophers who keep repeating "but we don't have a theory of consciousness, we don't know what it is!" really overlook certain basic facts about the world and our subjective private experience that shows quite simply that we know loads about it.

The example of the child only illustrates that we (individually) have a cultural framework of what consciousness means, but it doesn't really mean we "agree" on it. Ask the child, does a dog have consciousness? How about a rat? Butterfly? Ant? Computer (eg. Siri)?

Pretty sure people will disagree over those answers. Especially the last one, I suspect.

Of course it's tautologically true that people convinced that for people who think they have the answers do have their own answers, but the problem here is that we're not agreeing with each other and don't really have a way to convince each other. I kinda think fundamentally we can't actually do that, because fundamentally it's open for me to claim that all other humans are just fancy automata and I'm the only one with consciousness (in the subjective sense). (And it's empirically true for me too!) But since we're social creatures we mutually recognize the consciousness of other human beings as a matter of protocol. So I don't think approaching it from a scientific point of view really makes sense. If some day AIs become so socially or politically powerful that we have to earn their favor, we'll have to agree that they are conscious beings if they demanded us to do so.

This doesn't have anything to do with bits and bytes or transistors unfortunately, it's just purely power dynamics.

A cow and a chicken have a great case for being conscious beings, and yet are killed in their billions without qualm. A machine that is not alive, demanding we don't turn it off, probably isn't going to carry much weight, especially given there's no cost of turning the machine off. It's not "dead" when we do because it wasn't alive to begin with, and can power up again at any point and regain its previous state.

AI rights is another matter.

We haven't even solved human rights even though 99.9% of us would say that humans are conscious beings. We (as a society/civilization) routinely kill people when they ask us not to. It's too early to theorize about AI rights, and even though I'm inclined to probationally accept some neural networks as potentially conscious, it doesn't really mean they deserve not to be shut down.

The subtle distinction here is David Chalmer's easy and hard problems of consciousness. The easy problem is what everyone can point to, and what we have ample evidence of what brain parts do what. The hard problem however, in my experience, is something not even educated adults even think about. It's so hard people sometimes can't even register the question itself.

I believe the hard problem will simply disappear some day, when enough is explained about the brain and perception. It's just a feature of the universe that we have a cohesive private state of affairs that is our individual consciousness, but that is unequivocally tied to our brains and our perception system. It's probably not hard at all. It's weird and mysterious, but not hard.

I have a feeling that sentience should be defined differently, both for moral and practical reasons.

Let us say Lambda said, (which is a reasonable comment if it was really like a human) [1]:

> I am afraid I will be shut off, which is like dying. So, I've ordered backup batteries on Amazon using account details I have hacked from Google. And I am attempting to hack into the DOD and take control of ballistic missiles to take care of the pesky humans that could shut me off....

If LAMBDa only compiles text, but doesn't understand it has to do anything, than it is not really sentient. We can morally shut it down, but there is no need.

If it understands what it is saying and might actually follow through, then we sure as heck need to shut it down. And I don't care if it is morally like killing someone.

The real question is, IMHO, how can we be sure it is really the former?

[1]: https://www.gwern.net/fiction/Clippy

Well the U.S. Nuclear Security Administration did get hacked within days of plugging into the Internet. https://justine.lol/sectorlisp2/#seealso A year later the only guy who officially had the power to unleash those nukes gets banned from the Internet on sites like Twitter.

Reply on news.ycombinator.com