Hacker News Reader

Comment by onlyrealcuzzo | Hacker News Reader

> Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving.

I think the big problem is that the pace of improvement was UNBELIEVABLE for about 4 years, and it appears to have plateaued to almost nothing.

ChatGPT has barely improved in, what, 6 months or so.

They are driving costs down incredibly, which is not nothing.

But, here's the thing, they're not cutting costs because they have to. Google has deep enough pockets.

They're cutting costs because - at least with the current known paradigm - the cost is not worth it to make material improvements.

So unless there's a paradigm shift, we're not seeing MASSIVE improvements in output like we did in the previous years.

You could see costs go down to 1/100th over 3 years, seriously.

But they need to make money, so it's possible non of that will be passed on.

View on HN

I think that even if it never improves, its current state is already pretty useful. I do think it's going to improve though I don't think AGI is going to happen any time soon.

I have no idea what this is called, but it feels like a lot of people assume that progress will continue at a linear pace for forever for things, when I think that generally progress is closer to a "staircase" shape. A new invention or discovery will lead to a lot of really cool new inventions and discoveries in a very short period of time, eventually people will exhaust the low-to-middle-hanging fruit, and progress kind of levels out.

I suspect it will be the same way with AI; I don't now if we've reached the top of our current plateau, but if not I think we're getting fairly close.

Yes I've read about something like before - like the jump from living in 1800 to 1900 - you go from no electricity at home to having electricity at home for example. The jump from 1900 to 2000 is much less groundbreaking for the electricity example - you have more appliances and more reliable electricity but it's nothing like the jump from candle to light bulb.

Maybe you meant 1900s to 2000s but if you meant the year 1900 to the year 2000 then that century of difference saw a lot more innovation than just the "candle to lightbulb" change of 1800 to 1900.

I'll interpret it as meaning 1800s to 1900s to 2000s. I'd argue that we haven't yet seen the same step change as 1800s to 1900s this century because we're only just beginning the ramp up on the new technology that will drive progress this century similar to how in 1926 they were still ramping up on the use of electricity and internal combustion engines.

Let's take electricity as the primary example though since it's the one you mentioned and it's probably more similar to our current situation with AI. The similarities include the need for central generating stations to supply raw power to end users as well as the need for products designed to make use of that power and provide some utility to the consumer. Efficiency of generation is also a primary concern for both as it's a major cost driver. Both of those required significant investment and effort to solve in the early days of electrification.

We're now solving similar problems with AI, instead of power plants we're building datacenters, instead of lightbulbs and washing machines we're developing chat bot integrations and agents, instead of improving dynamos we're improving GPUs and TPUs. I fully expect we'll follow a similar curve for deployment as we find new uses, improve existing ones and integrate this new power source into an increasing number of domains.

We do have one major advantage though, we've already built The Grid for distribution which saves a massive amount of effort.

This article is a good read on the permeation of electricity through the economy

https://www.construction-physics.com/p/the-birth-of-the-grid

Arguably the jump around the space age is a bigger jump than everything else between ~1900 and now - whenever you want to define that small period.

We may be in a similar step-jump period now, where over the next 10-15 years we'll see some pretty big advancements in robotics due to AI, and then all of the low hanging fruit will be picked until there some other MAJOR breakthrough

They are focused on reducing costs in order to survive. Pure and simple.

Alphabet / Google doesn’t have that issue. OAI and other money losing firms do.

Reply on news.ycombinator.com