Hacker News Reader

Comment by semiquaver | Hacker News Reader

What universe do you live in that you can look at the past six months and see anything like a plateau in capability?

Edit: removed a comment that was uncharitable and rude, for which I apologize.

Most of the impressive accomplishments we’ve seen in the last few months have been the result of huge agent swarms working together and brute-forcing solutions, not massive leaps in intelligence from standalone models. That is still an improvement in the usefulness and power of the technology, but it is NOT evidence that model intelligence is increasing faster than before.

I don't have any access to any agent swarms (and neither do most) and i still think the models have obviously improved massively in standalone intelligence. Of course they have, agent swarms are not magic. You can swarm all you want around GPT-4 era models and you'll get nowhere. And i've never seen the term 'brute-force' more abused than these LLM discussions. Basically none of the results have been brute force.

Agreed. You can’t “brute force” reality, which has an infinitely large state space. A million monkeys won’t write Shakespeare and all that

"This machine-intelligence stuff is overrated, they are just using <insert particular machine-intelligence technique here>" isn't the resounding verdict it may have sounded like when you typed it.

Do you use them? I feel like you're talking about the news headlines. But in daily, direct, individual usage (one model at a time) I've found they've improved tremendously over the last few months, for tech/programming work at least

Yes, I do. They’ve definitely gotten better, I wouldn’t dispute that. But we haven’t gone from useless toys to Skynet in the last 6 months, progress is slower and steadier than that.

Not that they've hit it but that they are approaching it. The time to panic and steer the narrative is before you hit the iceberg, not after

People have been saying this since GPT-4.

Have we honestly seen that great a leap in the last 6 months, or just better application of what we had 6 months before that.

We are seeing multiple frontier models dropping on the same day and no one bats an eye, because it's more of the same.

The difference between 6 months ago frontier and now frontier in 3d modelling, graphics and video editing is night and day.

Just because there are new capabilities, doesn't mean they've pushed passed the plateau, they've just expanded where the previous solutions work.

We've gone from 80% in some places to 80% in some more places.

> We've gone from 80% in some places to 80% in some more places.

Any area that is verifiable will trend inexorably towards 100% over time. In unverifiable areas, it'll always be "80%" because the ubiquity of "AI" style erodes its value, and ">80%" for unverifiable things involves fashion, cachet and "vibes" that humans will probably never knowingly let it have.

Just checked your website, you really drank all the koolaid huh?

I don't think the math community thinks there's been a plateau. The models have gone from being a joke to being able to crank out proofs to research grade problems.

Statements like these remind me how much each of us are in our own unique information bubbles. People seemed very excited about analysing the differences of each model on my feeds.

Have you used recent ones on any large projects or coding issues? They've improved tremendously lately in my experience, in terms of implementing non-trivial things, debugging, handling legacy/complex codebases, etc. I usually keep a text document of things that models failed to do/fix and recent ones have wiped it all away.

Reply on news.ycombinator.com