Hacker News Reader

Comment by uejfiweun | Hacker News Reader

Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI. So isn't that kind of a natural brake, where we have to wait for computing speed to catch up with our ambitions? Why do we need additional handicaps and slowdowns on top of that?

I've seen a theory that the big tech companies are the ones pushing for regulation in order to achieve regulatory capture of this powerful new technology, which seems like a compelling theory indeed...

View on HN

Yup Big tech, which is really big advertising if you take Google+Meta online ad market share, get their main revenue from Likes/clicks/links/views counting + personal data. As info exploded you needed this crap to work out what to show ppl.

And ppl have learnt how to accumulate status/power/cash by gaming those numbers.

Chat-GPT et al dont rely on any of these number. So all the infrastructure of the Advertising-Industrial Attention capture complex can be thrown in the sea. Which should have been done long ago but we got free email, video, chat, games, movies etc so we gave away attention for free.

Behind the morons talking abt "god like" ai and similar ridiculous bullshit are mainly ppl worried about how they are going to capture the chimp troupes attention(and through it cash/status/power), if the tools they have relied on arent going to work anymore.

Eventually AI startups will end up where Google and Facebook ended up cause how else to fund their server farms? Subscriptions are never enough. Just ask Newspapers or Netflix.

> I've seen a theory that the big tech companies are the ones pushing for regulation in order to achieve regulatory capture

This... would track with what's historically been the case. Damn, that's an eye-opening take, thank you.

>Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI.

We don’t know that. It would seem likely, perhaps, but we don’t know that.

> Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI.

I don't see how that is the case at all. The human brain only needs about 100 watts. A GPT-3 instance only needs a few GPUs and maybe 1000 watts. GPT-4 is presumably in the same ballpark, let's say 10000 watts to be generous (even though it's probably closer to 1000 too), but still small enough to fit inside a full height rack.

Many datacenters are megawatt-level. Seeing how GPT-4 is already human-level at many tasks and watt-to-subjective-capability increases seem to grow linearly at worst, an AI could be ran in a single data center and perform 100x GPT-4 abilities. I'm not sure what 10x, much less 100x GPT-4 abilities would be, but that sure sounds like godlike or near-god to me.

Simply scaling up the current LLM models will not fix the fundamental problems of hallucination and lack of planing abilities. If they cost $10M+ to train, will scaling that cost 100X (to $1B) really make sense. I find it doubtful that the ROI is there

Consider that GPT4 is 15-30X the cost of GPT3.5, is it really 15-30X better? Not in my experience. I would expect that we are approaching the point of diminishing returns with text based LLMs, at least for the current architectures and training regimes. I'm interested to see what the Large Multi-Modal Models produce, but I suspect the planning & lying will still be there.

Chaining these models will increase the cost even more, without certainty that it will produce usable results for anything beyond simple, context limited tasks.

When it becomes that large and expensive, it makes sense for governments to do it and open up the result so everyone can benefit.

That also makes sense when we account for how gov'ts are really good at wasting money for below average results...

This is not something I would trust them to do competently or within a reasonable budget.

If it was done through the NSF or DARPA, with an industry partner, maybe? But I can't imagine the current politicians and them arguing about if & how to put restraints on it. It will be interesting to see how the CCP goes about this. Maybe the EU can offer a third alternative?

> That also makes sense when we account for how gov'ts are really good at wasting money for below average results...

The goal of governments is not efficiency, but fairness and strategic action aiming at long-term results.

I like to joke that one of the key goals of NASA is to prevent rocket scientists from moving to North Korea.

15-30x more expensive doesn't necessarily mean it actually requires 15-30x more computation. I'm certain GPT-4 is extremely profitable; they can charge whatever they want because nothing else like it exists. 3.5 turbo also isn't GPT-3/Davinci, which is still more expensive and often better for non chatbot functions. Additionally, 4 definitely provides at least 15x more value for me than 3.5. Turbo is a toy by comparison. The differences on complex tasks are subtle but profound.

OpenAI just received $10B and had to pause new paid subscriptions because more people were signing up than their hardware could handle. Even if they think it is diminishing returns they can afford to try.

Reply on news.ycombinator.com