Hacker News Reader

Comment by 0000000000100 | Hacker News Reader

I can't personally. OpenAI's o3 aside, the rate of progress in the past two years has been eye watering to say the least.

It's tricky since the future of AI isn't something anyone can really prove / disprove with hard facts. Doomers will say that the rate of improvement will slow down, and anti-doomers will say it won't.

My personal believe is that with enough compute, anything is possible. And our current rate of progress in both compute and LLM improvement has left Doomers with shaky ground to discount the eventuality of an AGI being developed. This just leaves ASI as a true question mark in my mind.

View on HN

> rate of progress in the past two years

This took me down a memory lane:

- Dragon Dictate speed recognition improvement curve in the mid-90s would have led to today's Siri sometime around 1999.

- The first couple of years of Siri & Alexa updates...

- Robots in the '80s led us to believe that home robots would be more or less ubiquitous by now. (Beyond floor cleaners.)

- CMU winning the DARPA Urban challenge for autonomous vehicles was a big fake-out in terms of when AVs would actually land.

Most of the benefits of computing come from relatively small improvements, continuously made over many years & decades. 2-4 years is not enough time to really extrapolate in any computing domain.

> with enough compute

"enough" here could be something that is only measurable on the Kardashev scale.

Wouldn't there be so many more possible futures though? Geopolitical conflict, economic crisis, the clime crisis, civil strife, demographic collapse, the end of globalization, an unexpected black-swan event, etc. Any one of these, even a pandemic, could push back, if not utterly prevent us from reaching these.

Endless growth and technological improvement isn't the only option, and seems to me like the least likely. The other option means that there will be a peak somewhere.

Very true and prescient. All of the technological growth of the last two decades has only been possible because of peace and cooperation between the Core Countries, but that world is as its most unstable point in decades and future peace is not guaranteed. As impressive as LLMs can be, a computer still loses to a crude home-made bomb.

> past two years has been eye watering to say the least

Are we seeing the same progress? GPT-4 was released in March 2023, that's almost two years. Tools are much better but where is the vast improvement?

I legitimately dont know how to reply, bc by this point llms co-own all aspects of my life and jumps between gpt4->claude3->claude3.5->o1 have all been very noticeable

I'm the opposite. We're presumably in a similar line of work, but while I've experimented with every major release from OpenAI and Anthropic last year -- I've barely ever used an LLM outside of that.

I still Google things I want to know and skip the AI part.

> I still Google things I want to know and skip the AI part.

My Google use is down significantly. And I mostly reach for it when I am looking for current information that LLMs do not yet have training data for. However, this is becoming less of an issue as of late. DeepSeek for example has a lot of current data.

GPT-2 was generating snippets of HTML ten years ago. Was it valid? Not always, but neither is the current crop. It's been incremental logarithmic gains approaching an asymptote for ten years now. Since before "Open"AI stopped being open.

GPT-1 was released 7 years ago, but ok. You really think GPT-4 to o1 is increasingly logarithmic the same way 4 to 4o is?

The rate of improvement in the models is nothing short of phenomenal, but the applications are meh at best, even after a few years of billions of dollars and endless hours poured by the world's best product and engineering minds. Every AI leader is pushing "agentic AI" as the next big thing but as a specialist in business automation I have my reservations. A lot of problems in automation in business happen because of insufficient investment in IT but can be solved fairly economically by off-the-shelf software, Zapier, a custom web service, or traditional ML techniques in order of complexity. Out of the more difficult problems that remain at the edges, only a small fraction can be solved by LLMs in my experience. The idea that chains of small agents will be composed and generally applied to any business problem under the sun doesn't sound right to me. I think not even the big bosses in AI know at present, but they're surely betting the house on it, and if the bet doesn't play out, things will start looking even more desperate.

> My personal believe is that with enough compute, anything is possible.

Dunno, we're already at ridiculous amounts of compute and progress has slowed, a lot. I think we need another technological breakthrough, a change in technique, something. LLMs don't seem to be capable of actually learning in the way humans do, just being trained on data, of which we've reached the limit.

Reply on news.ycombinator.com