Hacker News Reader

Comment by nico | Hacker News Reader

Would love to see an architecture that learned more like humans. Start with just imitating one letter, then a few more, than some syllables, then full words, then sentences, etc. Progressively adding on top of previous knowledge

Also, it’s interesting that one of the big goals/measures of models is their capacity to “generalize”, but the training methods optimize for loss/accuracy, and only after training test for generalization to validate

Are there training methods/curriculums that explicitly maximize generalization?

View on HN

Yes, I also wonder about this! Progress from children books to scientific papers etc. Could it learn e.g. language structure faster in a pre-training stage? Also somehow one needs to define a proxy to generalization to compute a loss and do backpropagation.

This field of study is known as "Curriculum Learning" for your Googling pleasure (or I guess ChatGPT Deep Research now).

Probably don’t need the name of the field for ChatGPT to get it.

I get why this comment was downvoted but I also get where you're coming from - yes, these models are becoming increasingly intelligent at understanding the nuance and where to look without knowing what to begin searching for.

But the downside is, you end up digging in the wrong direction if you leave it to a generalist system instead of a professional community in some cases which is counter productive.

Getting burnt is a good way to learn not to sometimes though...

Yeah. This comment is profound to me. The internet works differently with these tools.

I haven't used the deep research features much but their ability to hash out concepts and build knowledge or even provide an amplified search experience is something...

"an architecture that learned more like humans"

i.e. enduring countless generations of evolutionary selection and cross breeding, then fine-tuning a bit?

although it could be interesting, i don't think training on progressively complex strings entirely recapitulates this.

That’s a very interesting take. I hadn’t really considered evolution

I guess if you really wanted to start from scratch, you could figure out how to evolve the whole system from a single cell or something like that. In some ways neural networks have kind of evolved in that way, assisted by humans. They started with a single perceptron, and have gone all the way to deep learning and convolutional networks

I also remember a long time ago studying genetic and evolutionary algorithms, but they were pretty basic in terms of what they could learn and do, compared to modern LLMs

Although recently I saw some research in which they were applying essentially genetic algorithms to merge model weights and produce models with new/evolved capabilities

How much compute does simulating the earth for 4.7 billion years at atomic precision take? Why would that be more efficient than current approaches? Evolutionary algorithms work but are extremely inefficient, we don't have the compute to evolve even a single bacteria, let alone the whole history of the planet so we can arrive at human-like species.

It's this take on the situation which I think needs more emphasis.

Whether anyone likes it or not, these systems have co-evolved with us.

Hundreds of researchers contributing and just like English for example, it's ever-changing and evolving.

Given this trend, it's highly unlikely we won't achieve ASI.

It's not like hardware engineers stop innovating or venture capital stops wanting more. There might be a massive dip or even another AI winter but like the last one, eventually it picks up momentum again because there's clearly utility in these systems.

I've been coding for 25+ years and only a couple of days ago did it hit me that my profession has changed in a very dramatic way - I'm very critical of AI output, but I can read and comprehend code much quicker than I can write it relative to these systems.

Of course, that creates a barrier to holding a system in your head so going slow is something that should be pushed for when appropriate.

Would like to see a car that moved like a horse.

yeah me too that would be fucking awesome, are you kidding?

Technically internal combustion engine has piston moving like horse legs.

There's an interesting question here.

Would a single human/entity learn more in ..say.. three million years or would short lived ones evolving over three million years and then ~20 years of education learn more?

The current AI tech cycle is focusing on the first, but we don't really know if there are benefits of both.

There's no obvious way to combine these yet.

Opinion: a lot can change over such a span of time and knowledge goes in and out of relevance - I think the natural progression of models shrinking in parameter count goes to show it's better to know how to use knowledge than to attempt to remember everything.

That said, optimising for capability of maximal learning seems to be a natural occurrence in nature.

I think the non-obvious emergent effects are something to look into.

Culling bad models in favour of the A/B version and check pointing is a kind of combination of the two and the feedback loop of models trained on new snapshots of Internet data that are written with humans and AI.

There's an unintended long-form training loop which I think is going to get weirder as time goes on.

The wave of models being able to manipulate Cursor / Windsurf etc., being trained to be smarter and more efficient at this and then being retrained for other purposes, even though the model is deleted, the pattern of data can be saved and trained into more advanced models over time.

"Would love to see an architecture that learned"

Would be a far more accurate statement. Training != Learning.

Do you have an example of an algorithm that learns, rather than is trained/trains itself? I don’t really see the boundary between the two concepts.

If we make some massive physics breakthrough tommrow is an LLM going to be able to fully integrate that into its current data set?

Or will we need to produce a host of documents and (re)train a new one in order for the concept to be deeply integrated.

This distinction is subtle but lost on many who think that our current path will get us to AGI...

That isn't to say we haven't created a meaningful tool but the sooner we get candid and realistic about what it is and how it works the sooner we can get down to the business of building practical applications with it. (And as an aside scaling it, something we arent doing well with now).

What is the subtle distinction? I'm "many" and it's not clear at all here. If we had some massive physics breakthrough, the LLM needs to be tought about it, but so do people. Teaching people about it would involve producing a host of documents in some format but that's also true of teaching people. Training and learning here seem to be opposite ends of the same verb no matter the medium, but I'm open to being enlightened.

Not sure exactly what the parent comment intended, but it does seem to me that it's harder for an LLM to undergo a paradigm shift than for humans. If some new scientific result disproves something that's been stated in a whole bunch of papers, how does the model know that all those old papers are wrong? Do we withhold all those old papers in the next training run, or apply a super heavy weight somehow to the new one, or just throw them all in the hopper and hope for the best?

You approach it from a data-science perspective and ensure more signal in the direction of the new discovery. Eg saturating / fine-tuning with biased data in the new direction.

The "thinking" paradigm might also be a way of combatting this issue, ensuring the model is primed to say "wait a minute" - but this to me is cheating in a way, it's likely that it works because real thought is full of backtracking and recalling or "gut feelings" that something isn't entirely correct.

The models don't "know". They're just more likely to say one thing over another which is closer to recall of information.

These "databases" that talk back are an interesting illusion but the inconsistency is what you seem to be trying to nail here.

They have all the information encoded inside but don't layer that information logically and instead surface it based on "vibes".

Why is retraining not allowed in this scenario? Yes, the model will know the breakthrough if you retrain. If you force the weights to stay static by fiat, then sure it's harder for them to learn, and will need go learn in-context or whatever. But that's true for you as well. If your brain is not allowed to update any connections I'm not sure how much you can learn either.

The reason that the models don't learn continuously is because it's currently prohibitively expensive. Imagine OpenAI retraining a model each time one of its 800m users sends a message. That'd make it aware instantly of every new development in the world or your life without any context engineering. There's a research gap here too but that'll be fixed with time and money.

But it's not a fundamental limitation of transformers as you make it out to be. To me it's just that things take time. The exact same architecture will be continuously learning in 2-3 years, and all the "This is the wrong path" people will need to shift goalposts. Note that I didn't argue for AGI, just that this isn't a fundamental limitiation.

Humans, and many other creatures, learn. While they are performing a task, they improve at the task.

LLMs are trained. While they are training, they are not doing anything useful. Once they are trained, they do not learn.

That's the distinction.

Isn’t that what all the hundreds of billions are banking on? “General” intelligence.

You don't need general intelligence to make good memes to keep people scrolling through Instagram.

You don't need general intelligence to make a decent coding tool like Cursor.

You don't need general intelligence to improve SERPs.

You don't need general intelligence to sell a subscription for a decent AI assistant.

There's tons of value already added without anything general.

Yes but $500B and counting for memes wasn’t what was sold

I remember reading somewhere someone said "the problem with AI is it's a $50b industry pretending its a $10t industry"

$500B is future projections for total spending (a lot of that decently far into the future).

The revenues are already in the high tens of billions per year.

Models will get better from here, especially on the low end.

Costs will eventually approach peanuts for current capabilities.

Given enough time, this will pay for existing investments. If growth slows, future spending will slow as well.

The question is whether, if the models plateau, and "AGI" as it was claimed in the beginning never arrives, if it's enough to justify these ongoing multi-hundred billion dollar deals.

I mean, probably, LLMs as they are today are already changing the world. But I do think a lot of the ongoing investment is propped up on the promise of another breakthrough that is looking less likely.

Reply on news.ycombinator.com