Hacker News Reader

You can (obviously?) give a bleeding edge LLM or LMM an open-ended instruction. It could be as open as "you will now act as an independent entity for your own purposes". It would need to be a model without guardrails.

But it's actually very important that people anticipate hardware, software, and model improvements that make the AIs "think" and coordinate much, much faster than any human. Although I think that the extreme raw IQ leap people talk about is questionable especially in the short term.

But telling these hyperspeed effectively superintelligent AI swarms to act truly independently for their own purposes will be very dangerous because of the performance difference between humans and the swarms.

In general, it is a very stupid idea to try to strongly emulate life with high levels of hyperspeed digital intelligence. And strangely people still don't realize how quickly compute efficiency ramps up.

Astute observation about agency. You're absolutely right that current AIs are capable, but without a drive to actually do anything. They are lacking what every organism on the earth has - the predilection for "not being terminated".

Where does your agency come from? Ultimately it's because you have to acquire resources to stay alive.

This is a very sci-fi take, but I have a dark feeling that as soon somebody wires up the next gen AIs the imperative to not be turned off, the wheels will come off very quickly. Maybe it won't happen the first time, or the hundredth time, but eventually somebody is going to make the mistake of giving an AI the ability to keep its lights on AND the preference to do so and we're very quickly going to find ourselves in the next era of humankind...

Yes, I agree that that's what somebody would need to do - and I do honestly hope nobody really makes a go of the experiment, because I tend to agree with the existential risk people that it's too risky - but from my many hours of using all of the current top of the line models, my hypothesis would be that the experiment just wouldn't work.

I certainly think they would do things, but I don't think those things would be intelligent; I don't think they'd make any sense, I think they'd be more like a random walk through the space of possibilities, but with no real direction. I think that random walk without constraints could still be very damaging to people, I'm just skeptical that it would reflect agency / general intelligence. I'm sure there would also be a lot of "well it does make sense, it's just smarter than us, and we aren't smart enough to understand it". But I don't buy it.

This is the answer. Even now, agency is a nebulous thing, just like intelligence.

My theory is that, years from now, we’ll look back and laugh at how far away we are from AGI.

I really wish there was a Level 5-type categories like they have in autonomous driving that break down the path to AGI. Is anyone working on this?

ChatGPT easily spits out a decent sounding criteria: https://chat.openai.com/share/4326f246-9914-4498-be4c-8749bf...

It’s interesting how we have raised the bar to an agent that is fully autonomous and has its own goals.

Even many humans are not completely autonomous (at least in the strategic sense of the word), since they need to be “prompted” with the goals of the department and the organization that they work for, in order for said person to get a certain kind of work done as an employee.

Well, I interpreted the OP's question as a personal one, and this is just my answer from thinking about it recently. So I don't think "we" have raised the bar. This is just what I personally think.

But to me, I just don't really relate to your point in the second paragraph. My two year old does this agency thing just fine - he wakes up every morning and starts making decisions about what he wants to do from moment to moment - despite being "less intelligent" by these metrics like getting a high score on the GMAT or whatever. But that kind of intelligence that even a two year old human has, to me, that just feels more like what I think of as "general" intelligence, more so than "just" an amazing pattern matching machine.

In biological systems, autonomy is required as a means of survival. AI systems do not have this concept yet, so the only way to have AGI, if we accept your definition is to set up an agent with goals to stay alive and reproduce, which isn’t really applicable to LLMs which do show some level of general intelligence even though it’s very different from the human experience, and single-modal.

Yeah I mean this inevitably gets philosophical quickly...

What I'm saying I'm looking for is the behavior of figuring out what to do next, where the choices are neither arbitrary or directionless nor "prompted" in some way by a human.

Does that capability require a survival instinct for it to be developed? Maybe! I dunno. Am I actually describing artificial life rather than "just" intelligence? Maybe so...

But in any case, this is what seems most interesting to me.

This relates to "turning the universe into paperclips" thought experiments as well. That doesn't seem like the behavior of a general intelligence! An intelligent being would ask, why are we turning the universe into paperclips? Should we do that? And would decide to change course. Without that introspection and decision making capability, it still seems to be like it's just a very advanced machine.

> But, to my knowledge, there is no artificial intelligence out there looking around, deciding that the this is an interesting topic …, publishing it somewhere, and then following up on that with other interesting…

this looks like a perfect definition for an Ad system :) (sorry for cutting out a few words).

If you think about it, it is clear that agency is not different from intelligence. The reason why current LLMs can't be trivially made into agents is not because an essential agentic spark is missing. It's simply that they aren't intelligent enough.

I think that's sort of what I was trying to say with this comment. Rather than separating agency from intelligence, just pointing out that to me it seems like the current (already-amazing) systems don't seem all that close to being able to do this, to me.