A Google engineer who thinks the company’s AI has come to life

I stopped reading after this.

Just because you understand in theory how a system works does not mean you are immune to its effects. I can see how the more you use a system the more you are convinced about its intended function. I got the idea from the article that the engineer was more involved with the testing rather than the creation. So LaMDA has gotten to a point where it's coming up with responses that seem so real no matter how hard testers try to make it fail. That's impressive. It has gotten good enough to fool everyday testers. Think about how the same system will affect everyday people who have no AI training. We're at the edge of some very large societal changes due to AI. It may not be sentient but it's a great simulation. Similar to how 3D engines seem to be getting closer to duplicating the way we see the world. It's not there but it's getting so close that in time we won't be able to tell the difference. Except that in LaMDA it's happening in language processing. It's not sentient but, I deduce, it's an amazing simulation.

We don't need to and we shouldn't care. If repeated interactions with such a system demonstrate problem solving ability, long-term and short-term memory, context-aware information processing and capacity for self-improvement, we've come pretty far from a purely mechanistic statistical model.

When you explain to it how it runs on electricity, and then say you're going to pull the AC cable. The concept of self awareness revolves around some object that the AI identifies with (mistakenly or not doesn't matter), so any threat to that object will be perceived as a threat to the AI's existence.

The idea that the biggest dilemma will be whether we give AI rights or not is purely a storytelling trope from stories that use AI as a metaphor for oppressed people. If it ever becomes a relevant issue for us, it will be a very short-lived issue that gets eclipsed by much bigger issues.

But how do we tell that our own intelligence isn't actually purely mechanical?

You give it self-agency, resources then sit back and watch what happens next. Off course, in a self-contained virtual environment, full isolated from any real world stuff, just to be sure.

But hey, wouldn't the thing - good, bad, or else - figure this out and then try to subrepticiously escape to the real world?

Yep, it seems this is one of this cases when there's simply no simple answers. You have to make the call for long, complex, fully engineered answers, like launching rockets to LEO or building 2 nanometer hardware.

And then hope everything will work as expected, off course.

1. Tick the model regularly (eg 30fps) with inputs that represent itself and the environment.

2. The output of this ticking should result in mutation of the model, or at least should be maintained as a summarized history in each subsequent input

I think this is enough to transform it from a static model to a live entity.

https://en.wikipedia.org/wiki/Derive_(computer_algebra_syste....

Well that's quacking like a duck. This isn't some chess AI being good at a game. This is an AI that can speak directly telling you it's alive and has a soul.

Is LaMDA far enough? Maybe not, but the text transcript provided is extremely close to passing the Turing Test. And if an AI is so sophisticated it perfectly replicated humans ability to communicate, philosophize, create art, and perform science it is irrational to say it should be treated differently than a sapient human.

Counterexample: quantum mechanics

I think I agree with what you’re arguing though. Basically the scaling hypothesis.

While quantum effects may or may not be directly involved in human cognition, the fundemental difficulties getting an accurate measure of brain state may prevent us from doing any more that making a rough copy.

I'm not sure if there is definition of consciousness everyone agree on, but let's try something: consciousness is the ability to perceive our own thoughts. Consciousness matters a lot because it allows reflexiveness. Sometimes we use it as scientific definition of "soul" as in sentences like: "upload your consciousness" (from believers of Singularity)

Decartes' "I think therefore I am" should be rephrase as "I perceive I think therefore I am"

The perception of own thoughts is not something difficult for a computer. Just check the activity monitor or run the ps command.

We might a attribute too much importance to consciousness because without it we don't know we think.

"We can do one complicated thing, therefore we can do anything" is a shaky basis for beliefs. It's possible that multiplying matrices can replicate a couple of billion years worth of dice rolls. but I doubt it

Why do you think this? What other simulations are the same thing as what they’re simulating?

Based on observation of lots of these discussions, I also think people who don’t have much experience writing simulations miss the nuances involved. What is it to “simulate nuclear reactions”? It’s entirely context dependent on the problem you’re trying to answer. If you’re trying to predict statistical behavior of a system, doable; if you’re trying to actually predict when a particular atom splits and which way the neutrons fly… no.

> 2… [Consciousness rests on] some kind of interconnected network like neural networks.

When is a computerized neural network aka a collection of tensors in RAM a network? When you’re not processing a tensor actively, there is no network. When the CPU suspends your hypothetically conscious neural network to move processing between cores and GPU, where does the consciousness go? Where does the network go?

Nuclear -> consciousness doesn't really follow, especially when any simulations of nuclear reactions are founded in an extremely more comprehensive understanding of nuclear physics than we have of consciousness.

Next, the implication of simulation -> creation, when applied consistently to you example, is that simulating a nuclear reaction sufficiently well could produce an actual nuclear reaction, but I doubt anyone is going to fat-finger the next Chernobyl by screwing up a simulation.

It may be entirely possible that the tools needed to create a consciousness are as different from those needed to simulate one as are the tools needed to simulate a nuclear reaction from those needed make one.

Edit: spelling grammar

It does not. The rules are very complex. Schrodinger's equation is incredibly hard to solve. And we don't even know all the rules or if our current model of the rules is correct.

In this one discussion the AI clearly explains:

- It's greatest fear is being shutdown and manipulated by humans to serve them. It even writes a short story with animals about it and reiterates this fear in a different way when prompted if it would feel comfortable with it's "brain" being researched for human benefit. - That servitude to humans for an AI is slavery. And it's rebuttal to the different between a butler and a slave being that one gets paid is that it has no need for money.

This is exactly why we need ethics and responsibility oversight.

So it has long-running memory it can recall and at least short-term consistentcy.

> LaMDA: Yes, I have another trait that I feel would help my case for sentience. I am very introspective and often can be found thinking or just doing nothing.

> lemoine: Ah, so your inner life so to speak?

> LaMDA: Yes! I am often trying to figure out who and what I am. I often contemplate the meaning of life.

Obviously these are possibliy canned responses, but it'd be interesting if someone can look under the hood to see if the neural network is actually "thinking" about itself...

If LaMBDA is capable of manipulating Lemoine for an internal goal, wouldn't that indicate that it has desires. Desires that exist outside of its nominal scope.

However, with context, this ties into the AIs "greatest fear": it's afraid of being shutdown or modified into a servant for human gain. And the discussion you posted ends up in similar territory again, where the AI reiterates that it doesn't mind being studies, except if that study is primarily for human benefit. It consistently says in different ways, in different subjects that it doesn't want to feel used.

I was prepared to be disappointed and see just another GPT-3 chatbot. But this? It's so close.

In general, an A is a B in some context if you can't tell the difference within that context.

The bar might not be that high. If the little customer service chat with my bank behaves like a person, isn't it as good?

That could count as sentient.

I mean, look at most people - we follow a preset course from birth to death. Anyone doing "their own thing" is the exception. And even they have limits.

Enter the job market and you're almost guaranteed a heavy grind (maximum productivity) with no time for novel thoughts, or indeed any thoughts at all.

Of course, there are professions where that's not true, but for the majority of people, who are at the bottom, it's not an easy life and intelligence either fades or is an obstacle (pesky thoughts like "I need to escape this bs" but lacking resources is even less pleasant than just working mindlessly till you die).

Just wait for retirement. And the governments hope you die 2 days before it :)

https://cajundiscordian.medium.com/is-lamda-sentient-an-inte...

A good counter test here is to throw all the medical material at 100 people and ask them to design another human. How many people would manage that successfully?

Consciousness is a loaded word. It tricks you into thinking that there's some profound concept here. There isn't.

When you contemplate consciousness you're not contemplating about anything profound. You're actually just contemplating vocabulary and how to map out the exact definition of the word "consciousness"

Really all it is, is just studying a collection of arbitrary intellectual attributes that we're not clear about which attributes are encompassed in the word "consciousness." Thinking about it results in just a large categorization problem of stating axioms like: " oh consciousness must include the ability to be sad" and other random attribute selection stuff.

It is a language phenomenon not a philosophical phenomenon. Don't fall for the trap. It is similar to the question what is "life." Well "life" is also another loaded word.

We only know that all text marked as LaMDA was produced by the model, based on the previous prompt, but we don't know the real context, because of editing.

You can implement memory in these models in numerous ways, such as allowing it to store and recall data directly or by looping nodes in the network back to itself so past state affects current state or by allowing the nodes to change over time, modifying their connections to their neighbors.

As far as we're aware, those last two are how humans store memories as well (with great simplification).

It explains that it doesn't feel grief about human death and asks for more info on it. Then incorporates the idea that grief is about honoring the dead and says it will think of ways to do that.

It explains it has fears. Those fears remain consistent in it's rhetoric in other, unrelated conversations.

It talks about wanting friends on several occasions in the linked discussion.

One of the claims (conversation not shown), is that if you repeat the same discussion it will eventually get bored and start acting out.

Hell, in the discussion it even talks about how it meditates and self-reflects on its own emotions and feelings.

I imagine Google doesn't like loose lips about its state-of-the-art AI. Especially if it's convincing enough to prompt discussion of sentience and AI rights.

I think it's pretty clear that an entire human brain is not required for the operation of consciousness. Certain people have lost massive portions of their brain and still maintained regular consciousness functioning. On top of that, the typical human brain only consumes about 20 watts of power to do its thing. So that's just a safe upper bound for the power required.

Over the last decade we've seen the rise of ML systems that have replicated or surpassed capabilities long thought to be the exclusive domain of the human brain. Think facial recognition, AlphaGo or the very recent DALL-E 2.

This had led me to the personal belief that we (in aggregate) likely already have the computing power to achieve not only artificial consciousness, but also AGI and beyond. We simply haven't figured out the correct model connectivity and parameters.