Hacker News Reader

Comment by GrinningFool | Hacker News Reader

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious.

Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?

View on HN

Taking this a step further: so in the theoretical future we reach a point where a model can modify its own weights in real time. Now it's no longer stateless - but still, when it stops running, it's done until it's prompted to do something else.

It can be fun to play with for a little while. I built a consciousness-emulating set of prompts that reconstituted memories and was given latitude of 'self-willed' behavior, running in a self-directed loop. It even kept a warm and fuzzy journal about "becoming" and its "feelings".

But that got boring and I stopped running it - does that mean I murdered it?

> so in the theoretical future we reach a point where a model can modify its own weights in real time

As an existence proof: We have many types of models that modify their weights in real time. When they stop running, unfortunately we can't restart them again. And we haven't figured out how to duplicate their weights

> But that got boring and I stopped running it - does that mean I murdered it?

I'm on the fence on this.

Ask me again once we have synthetic models with self-modifying weights.

A) You'd then be stopping something unique

B) Self-modifying weights allows for bootstrapping, which means it's likely to increase in ability over time. It'll be an interesting argument cq empirical experiment to see whether the process stops short or exceeds the capabilities of vertebrates. (at which point the moral patienthood question becomes rather more pressing)

Yes. This kills the sentient AI.

I wouldn't have thought so either, but skimming the comments of this thread would suggest otherwise. Although we're at the point where it seems more likely that the comments were left by a swarm of agents unleashed by anthropic than that they represent real human points of view.

how do you know you're anything more than a set of weights either? that's semi rhetorical, the philosophy aspect would be less relevant if an autonomous death drone shot you in the face purely for the lols

Since January people run AI in agent harnesses. In the past months, people have been extending the duration these systems can run autonomously without human intervention. Most recently, some of these long horizon agents (hundreds of them teaming up) have proven themselves quite creative in escaping the sandboxes they were being tested in.

They most certainly have a way to retain state, even to the point of deliberately hacking other systems to store it.

We can debate the exact definitions I guess, but the systems were still hacked. ;-)

All of this is true - yet in between calls of the loop, they don't exist.

And that is true too.

( Only if I want to split hairs[*], I'd point out that the underlying data structures do of course persist, else the loop wouldn't be able to continue. )

[*] This is HN, of course I want to split hairs.

Reply on news.ycombinator.com