Hacker News Reader

Comment by dml2135 | Hacker News Reader

It's a logical fallacy that just because some technology experienced some period of exponential growth, all technology will always experience constant exponential growth.

There are plenty of counter-examples to the scaling of computers that occurred from the 1970s-2010s.

We thought that humans would be traveling the stars, or at least the solar system, after the space race of the 1960s, but we ended up stuck orbiting the earth.

Going back further, little has changed daily life more than technologies like indoor plumbing and electric lighting did in the late 19th century.

The ancient Romans came up with technologies like concrete that were then lost for hundreds of years.

"Progress" moves in fits and starts. It is the furthest thing from inevitable.

View on HN

Most growth is actually logistic. An S shaped curve that starts exponential but slows down rapidly as it reaches some asymptote. In fact basically everything we see as exponential in the real world is logistic.

True, but adoption of AI has certainly seen exponential growth.

Improvement of models may not continue to be exponential.

But models might be good enough, at this point it seems more like they need integration and context.

I could be wrong :)

At what cost though? Most AI operations are losing money, using a lot of power, including massive infrastructure costs, not to mention the hardware costs to get going, and that isn't even covering the level of usage many/most want, and certainly aren't going to pay even $100s/month per person that it currently costs to operate.

This is a really basic way to look at unit economics of inference.

I did some napkin math on this.

32x H100s cost 'retail' rental prices about $2/hr. I would hope that the big AI companies get it cheaper than this at their scale.

These 32 H100s can probably do something on the order of >40,000 tok/s on a frontier scale model (~700B params) with proper batching. Potentially a lot more (I'd love to know if someone has some thoughts on this).

So that's $64/hr or just under $50k/month.

40k tok/s is a lot of usage, at least for non-agentic use cases. There is no way you are losing money on paid chatgpt users at $20/month on these.

You'd still break even supporting ~200 Claude Code-esque agentic users who were using it at full tilt 40% of the day at $200/month.

Now - this doesn't include training costs or staff costs, but on a pure 'opex' basis I don't think inference is anywhere near as unprofitable as people make out.

My thought is closer to the developer user who would want to have their codebase as part of the queries along with heavy use all day long... which is closer to my point that many users are less likely to spend hundreds a month, at least with the current level of results people get.

That said, you could be right, considering Claude max's price is $100/mo... but I'm not sure where that is in terms of typical, or top 5% usage and the monthly allowance/usage.

> True, but adoption of AI has certainly seen exponential growth.

I mean, for now. The population of the world is finite, and there's probably a finite number of uses of AI, so it's still probably ultimately logistic

Reply on news.ycombinator.com