Hacker News Reader

Comment by surgical_fire | Hacker News Reader

> I am constantly surprised how prevalent this attitude is. ChatGPT was only just released in 2022. Is there some expectation that these things won't improve?

Is there any expectations that things will? Is there more untapped great quality data that LLMs can ingest? Will a larger model perform meaningfully better? Will it solve the pervasive issue of generating plausibly sounding bullshit?

I used LLMs for a while, I found them largely useless for my job. They were helpful for things I don't really need help with, and they were mostly damaging for things I actually needed.

> This is ego speaking.

Or maybe it was an accurate assessment for his use case, and your wishful thinking makes you think it was his ego speaking.

View on HN

> Is there any expectations that things will?

Seems like an odd question. The answer is obviously yes: There is a very pervasive expectation that LLM's will continue to improve, and it seems odd to suggest otherwise. There is hundreds of billions of dollars being spent on AI training and that number is increasing each year.

> Is there more untapped great quality data that LLMs can ingest?

Why wouldn't there be? AI's are currently trained on the internet but that's obviously not the only source of data.

> Will a larger model perform meaningfully better?

The answer to this, is also yes. It is well established that, all else being equal, a bigger model is better than a smaller model, assuming that the smaller model hasn't already captured all of the available information.

We recently had a few submissions about this topic. Most recently Ilyas talk. Further improvement will be a research type problem. This trend was clear for a while already, but is reaching the mainstream now. The billions of dollar spend goes into scaling existing technology. If it doesn't scale anymore and becomes a resarch problem again, rational companies will not continue to invest in this area (at least without the usual research arrangements).

> The answer is obviously yes: There is a very pervasive expectation that LLM's will continue to improve, and it seems odd to suggest otherwise. There is hundreds of billions of dollars being spent on AI training and that number is increasing each year.

That makes an assumption that throwing dollars on AI training is a surefire way to solve the many shortcomings of LLMs. It is a very optimistic assumption.

> Why wouldn't there be? AI's are currently trained on the internet but that's obviously not the only source of data.

"The Internet" basically encompasses all meaningful sources of data available, especially if we are talking specifically about software development. But even beyond that, it is very unclear what other high quality data it would consume that would improve the things.

> The answer to this, is also yes. It is well established that, all else being equal, a bigger model is better than a smaller model, assuming that the smaller model hasn't already captured all of the available information.

I love how you conveniently sidestepped the part where I ask if it would improve the pervasive issue of generating plausibly sounding bullshit.

The assumption that generative AI will improve is as valid as the assumption that it will plateau. It is quite possible that what we are seeing is "as good as it gets", and some major breakthrough, that may or may not happen on our lifetime, is needed.

> That makes an assumption that throwing dollars on AI training is a surefire way to solve the many shortcomings of LLMs. It is a very optimistic assumption.

That's not an assumption that I am personally making. That's what experts in the field believe.

> "The Internet" basically encompasses all meaningful sources of data available, especially if we are talking specifically about software development. But even beyond that, it is very unclear what other high quality data it would consume that would improve the things.

How about, interacting with the world?

> I love how you conveniently sidestepped the part where I ask if it would improve the pervasive issue of generating plausibly sounding bullshit.

I was not trying to "conveniently sidestep". To me, that reads like a more emotional wording of the first question you asked, which is if LLM's are expected to improve. To that question I answered yes.

> The assumption that generative AI will improve is as valid as the assumption that it will plateau.

It is certainly not as valid to say that generative AI will plateau. This is comparable to saying that the odds of winning any bet are 50/50, because you either win or lose. Probabilities are a thing. And the probability that the trend will plateau is lower than not.

> It is quite possible that what we are seeing is "as good as it gets", and some major breakthrough, that may or may not happen on our lifetime, is needed.

It's also possible that dolphins are sentient aliens sent here to watch over us.

> That's not an assumption that I am personally making. That's what experts in the field believe.

People invested in something believe that throwing money at it will make it better? Color me shocked.

"eat meat, says the butcher"

The rest of your answer amount to optimistic assumptions that yes, AI future is rosy, based on nothing but a desire that it will, because of course it will.

> There is a very pervasive expectation that LLM's will continue to improve, and it seems odd to suggest otherwise. There is hundreds of billions of dollars being spent on AI training and that number is increasing each year.

It isn't odd at all. In the early 21st century there were expectations of ever, exponentially increasing processing power. This misconception partially gave us the game Crysis, which, if I'm not mistaken, was written with wildly optimistic assumptions about the growing power of computer hardware.

People are horrible at predicting the future of technology (beyond meaninglesslessly or trivially broad generalizations) and even when predictions turn out to be correct, they're often correct for the wrong reasons. If we were better at it, even in the shortest term, where such predictions should be the easiest, we'd all be megamillionaires, because we'd have seen the writing on the wall and invested in Nvidia before the AI craze reached its current fever pitch.

> because we'd have seen the writing on the wall and invested in Nvidia before the AI craze reached its current fever pitch.

I did this when I saw the first stable diffusion AI titties. So far up over 10x.

If I had a nickel for every tech that takes off once porn finds its way to it, I wouldn't be counting in nickels.

What's the logic behind that? Porn has had nothing to do with the current AI boom.

It’s not worth arguing with anyone who thinks LLMs are a fad. I just tell them that they’re right and get back to using LLMs myself. I’ll take the edge while I can.

If all of this is true, then I would expect LLMs today to be in a whole different league of LLMs from two years ago. Personally I find them less helpful and worse at programming and creative writing related tasks.

YMMV but count me in the camp that I think there’s better odds that LLMs are at or near their potential vs in their nascent stages.

Reply on news.ycombinator.com