Hacker News Reader

Comment by TheAceOfHearts | Hacker News Reader

Do you think we will get AI models capable of learning in real time, using a small number of examples similar to humans, in the next few years? This seems like a key barrier to AGI.

More broadly, I wonder how many key insights he thinks are actually missing for AGI or ASI. This article suggests that we've already cleared the major hurdles, but I think there are still some major keys missing. Overall his predictions seem like fairly safe bets, but they don't necessarily suggest superintelligence as I expect most people would define the term.

View on HN

This is a condensed version of Altman's greatest hits when it comes to his pitch for the promise of AI as he (allegedly) conceives it, and in that sense it is nothing new. What is conspicuous is that there is a not-so-subtle reframing. No longer is AGI just around the corner, instead one gets the sense that OpenAI they have already looked around that corner and seen nothing there. No, this is one of what I expect will be many more public statements intended to cool things down a bit, and to reframe (investor) expectations that the timelines are going to be longer than were previously implied.

Cool things down a bit? That's what you call "we're already in the accelerating part of the singularity, past the event horizon, the future progress curve looks vertical, the past one looks flat"? :D

For Sam Altman, honestly yes. The man has said some truly wild things for the VC

If any of that were true, then the llm would be actively involved in advancing themselves, or assisting humans in a meaningful way in the endeavor, which they’re not so far as I can tell.

Maybe Sam meant it more generally - humanity has achieved a lot over past thousands of years, and now we are finally close to interacting with systems that are more capable intellectually.

Artificial intelligence is a nourished and well educated population. Plus some Adderall maybe. Those are the key insights which represent the only scientific basis for that term.

The crazy thing is that a well crafted language model is great product. A man should be content to say "my company did something akin to compressing the whole internet behind a single API" and take his just rewards. Why sully that reputation boasting to have invented a singularity that solves every (economically registerable) problem on Earth?

Such is the nature of the scorpion.

Because they promised the latter to the investors and due to the nature of capitalism they can never admit they’ve done all they can.

I hate to enter this discussion, but learning based on a small number of examples is called few-shot learning, and is something that GPT-3 could already do. It was considered a major breakthrough at the time. The fact that we call gradient descent "learning" doesn't mean that what happens with a well-placed prompt is not "learning" in the colloquial sense. Try it: you can teach today's frontier reasoner models to do fairly complex domain-specific tasks with light guidance and a few examples. It's what prompt engineering is about. I think you might be making a distinction on the complexity of the tasks, which is totally fine, but needs to be spelled out more precisely IMO.

Are you talking about teaching in the context window or fine tuning?

If it is the context window, then you are limited to the size of said window and everything is lost on the next run.

Learning is memory, what you are describing is an llm being the main character in the movie Momento, I.e. no longterm memories past what was trained in the last training run.

There's really no defensible way to call one "learning" and the other not. You can carry a half-full context window (aka prompt) with you at all times. Maybe you can't learn many things at once this way (though you might be surprised what knowledge can be densely stored in 1m tokens), but it definitely fits the GP's definition of (1) real-time and (2) based on a few examples.

Yes, one is committing knowledge to neurons, the other is commuting knowledge to short term memory.

Put another way, if you took a llm with random weights. Do you expect you could rely on context alone?

AlphaZero learned various board games from scratch up to better than human levels. I guess in principle that sort of algorithm could be generalized to other things?

What you described can be (and is being) achieved by agentic systems like Claude Code. When you give it a task, it knows to learn best practices on the web, find out what other devs are doing in your codebase, and it adapts. And it condenses + persists its learnings in CLAUDE.md files.

Which underlying LLM powers your agent system doesn't matter. In fact you can swap them for any state-of-the-art model you like, or even points Cursor to your self-hosted LLM API.

So in a sense every advanced model today is AGI. We were already past the AGI "singularity" back in 2023 with GPT4. What we're going through now is a maybe-decades-long process of integrating AGI into each corner of society.

It's purely an interface problem. Coding agent products hook the LLM to the real world with [web_search, exec_command, read_file, write_file, delegate_subtask, ...] tools. Other professions may require vastly more complicated interfaces (such as "attend_meeting",) it takes more engineering effort, sure, but 100% those interfaces will be built at some point in the coming years.

Reply on news.ycombinator.com