Hacker News Reader

Comment by gmaster1440 | Hacker News Reader

A lot of people appear to struggle with the likely possibility that there is both a lot of AI hype to go around and it's still a very disruptive technology that will improve a lot over relatively short timelines.

View on HN

I remember in 1999 or so near the top of the dot-com bubble, there was similar hype, and I was like "Sure, over 20 years this thing is going to be huge. Over the next 2, many people are going to get screwed." Same thing for crypto in 2014 and then again every time a new crypto bubble happens.

The problem is that capital isn't that patient. People are sinking billions into LLM integrations, at discount rates of 5%+. If it takes 20 years for a tech to pay off, at a 5% discount rate, and you sunk a billion dollars into it, it needs to earn back $2.65B. Moreover, at some point during that 20-year period, somebody's going to ask "Where's my billion dollars?" and pull the plug on the project.

I think the tech behind LLMs may eventually be game-changing, but that tech is going to change ownership and get reinvented several times, and we won't actually see profitable, sustainable benefits until industries get refactored into cost structures that make sense for what LLMs can actually do. The web needed its Netscape and Yahoo and Geocities and Apache, but it also needed Google and Rails and Django and GitHub and Stripe and Facebook and Webkit and nginx and MySQL to really become what it did.

> what LLMs can actually do

You should see them as a passage - something in study and development today for something more complete tomorrow.

I would agree except the gap between what's realistically cool about LLMs (make tons of traditional NLP tasks easier/better) and what people are hyping them up to be (potentially world ending sentient beings) is so tremendously large it essentially guarantees an AI winter despite the fact LLMs represent a major advancement in practical SotA.

It also does concern me how basically nobody is building a real product around the current state of "AI", but are rather hinging their success on what they believe it will be in the near future.

> what's realistically cool about LLMs

This morning I met an newspaper article: "What will be the results of the European elections? Let us ask ChatGPT".

Newspapers have a long-standing tradition of exactly the same disconnection from reality that LLMs get flak for.

https://benwheatley.github.io/blog/2024/03/19-14.53.05.html

Very nice article, Ben.

But see, being it a given from experience that humans can be so bad in judgement, reasoning, professionalism, output... Why did we strive for superhuman judgement, reasoning, professionalism, output? (The same way we strived for superhuman strength.)

Thanks.

> Why did we strive for superhuman judgement, reasoning, professionalism, output?

Lots of possible reasons; there's many ways for it to be valuable.

(And it's not like the newspapers are deliberately writing fiction, with notable exceptions like The Onion).

If it's a rhetorical question, I'd be interested to know what you had in mind :)

Although it seems looking at the article it said there were significant gains for right-wing parties, so it did end up being right on that one!

If ML froze for the next ten years, we'd still be integrating everything we'd have today. The current progress of today has already reached a minimum threshold of quality.

100% - the current SOTA with the right implementation can already replace a lot of contact center work, give us a real-life J.A.R.V.I.S. and make video games where your decisions actually influence the story in a wildly meaningful way.

I've noticed that some korean restaurants are already doing really excellent customer ordering agents over the phone.

I tend to agree with the caveat that we are experiencing exponentially diminishing marginal returns on investment (energy, FLOPS, currency). Yes it's going to get a lot better and smoother to work with. Agents will become secret and someday very soon no one will want to hype that they are using the tech. But we will need another revolution to get the kind of phase change we experienced with the introduction of LLMs. Imo the biggest thing that will lead to improvement now is open models that will get reused and remixed in creative ways.

edit: to be clear, I'm saying that we are running into scaling "walls" that make hard extrapolation based on increased investment senseless.

Gartner puts Generative AI at just about it's peak in terms of Hype back in August of 2023[1], with a 2-5 year time frame to go through the "trough of disillusionment" and then back to the "plateau of productivity", which seems pretty fair to me.

[1]https://www.gartner.com/en/articles/what-s-new-in-the-2023-g...

The disruptive technology standard trend of: "the market would be huge" to "there is no market" to "there is a market, and it's growing"

I suspect that the success of AI will also vary a lot between different applications. But most people will evaluate AI on the topics they're most familiar with.

It will likely be disruptive in some areas short term, but will take much longer in other areas.

The problem is that there is little, incremental progress last 1 year or so after the big chatGPT boom to justify the hype, technically wise. Most of the "progress" going on is basically marketing, and making the models respond in ways humans like, or being more useful in certain practical applications. The basic, fundamental issues/limitations remain unanswered and unaddressed. As products, they have improved a lot and most probably are gonna improve more. But if we are talking for going towards AGI or more complex applications, I do not see evidence on that except as toys.

I see it being used a lot now. AGI is a mirage and I think LLMs have made this more clear. We will never get there because the goal posts will always change. We will eventually get to the point that the prenatal experience is viewed as critical to AGI.

What complex applications are you speaking of?

It us always interesting to see these takes.

The past year has brought us both model improvements along with drastic cost reductions. Its been a pretty magical year imo.

Is it AGI? Not even close but we have been utilizing the tooling improvements to build products internally.

Hardware improvements alone provide a straightforward path to a substantial improvement over what we have now, and we should see software improvements and more data to use over time as well. I'm not quite at "singularity" level of hype but this tech will just get used more and more and more.

It's one of those short term is overhyped, long term is underestimated things.

The main reason LLMs may appear disruptive at this point is because the real cost has yet to be fully factored in.

Not too many applications can justify spending huge amounts of energy generating answers that may just be nice sounding BS.

The main cost is training rather than inference, and compute is still getting cheaper, so I don't expect this to limit the use of LLMs.

We may finally see people realising that just because they can use a hammer on a screw, doesn't make it the best choice.

cheaper, but still crazy damn expensive

it makes sense because big money is betting it's worth the investment -- but it may not be

I'm not so certain. You can already run a GPT-4-quality model locally on a decent desktop, and GPT-3-quality models on low-powered chips - plus data centers will benefit from scale. A lot of third-party services are using paid APIs that (based on cases like Mistral where some models are publicly available) appear to more than cover the inference costs.

There are also plenty of uses for LLMs beyond generating hopefully-accurate answers, such as for fictional content or use as foundation models for tasks like translation. Though we are definitely in the "throw things at the wall and see what sticks" stage currently.

Yup. Humans are ironically (alazṓn-ically?) very binary when it comes to classification of things.

Even more people seem to struggle with the concept that it may be underhyped.

Ive been working with them for a while now and the areas I see it disrupting are:

* User interfaces. It does provide a genuinely new way to interface seamlessly with existing software.

* Translation.

* Generating bullshit. Yes, there's demand for this even if perhaps there shouldnt be.

* Not much else. That includes using it as a specialized autocomplete. I think it falls down pretty badly at that.

It is (or was, see my other comment) pretty good at generating standardized boilerplate code, which was quite useful.

Not so good at anything that's (for want of a better term) "creative".

Ive noticed that too. I think it will lead to an excess of boilerplate and less refactoring to remove boilerplate. I'd consider these things to be at best neutral and maybe net negative.

RAG?

I filed that under "new kind of UI".

Reply on news.ycombinator.com