Hacker News Reader

Comment by anthonypasq | Hacker News Reader

Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.

This may just be a classic case of Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox

In short, better hardware will drive down token cost in the near-term, but will drive up the demand for tokens as it gets cheap enough for other sectors to start to use it heavily.

It comes from steam engines where economists originally thought that coal demand would plummet with more efficient engines, but it actually just meant that we found more uses for steam engines.

This is exactly what I see happening now.

Codex keeps doing these usage resets. What do I do? Burn even more tokens than ever before. I know I'm not the only one.

Is this a normal thing now? I remember seeing this talked about as a surprising thing, but now I'm seeing posts about this as if it's normal.

(I switched to using local models as usage limits, api instability and the concept of paying per token stresses me out)

the total cost spent on tokens may go up, but i just cant imagine per token costs going up

Depends on compute capacity. If we become supply constrained on tokens, then prices will necessarily go up.

no they dont because inference stacks are getting more efficient and models are getting more intelligent per parameter.

I would posit there’s no way in hell they’re getting sufficiently cheaper on a short enough time frame vs how much demand is sky rocketing. AI companies are seeing quarterly doubling of revenue if not more.

If we are applying Jevons paradox to this then the unit being consumed is not tokens but the inputs for token production - power, capex, something else. To draw an analogy to the steam engine, coal:electricity::mechanical-work:tokens. Jevons paradox does not talk about mechanical work becoming cheaper in the short term setting up a sort of rubber band of demand creating spiking prices for mechanical work. Compared to the renaissance, mechanical work was much cheaper throughout the industrial revolution and remains cheaper to this day. We can still definitely say that the easier it is to produce tokens, the cheaper they will be.

All Jevon’s paradox says is that as a resource becomes cheaper total consumption of that resource increases. It applies equally well to the inputs of token production as it does to the tokens themselves. The former would describe the effect the sellers into AI companies see (energy, GPU chips, RAM etc - if they lower their prices they’ll have more overall consumption) while the latter describes what the AI companies see with their customers (if they lower token prices consumers will use more tokens overall).

Jevons paradox states it might increase. It is not an ironclad law and there are many many cases where increasing efficiency wrt. a certain resource really will decrease the total consumption of that resource. Yes tokens are an input themselves, but this thread is discussing hardware that is more efficient at generating tokens. To increase the efficiency by which tokens are converted into some other product would require innovation in some other area - harnesses, the models themselves, better skill from the prompters, etc.

I think you're reducing a very complex thing (the global economy) into a very simplistic model (Jevons' paradox) and thinking both are the same thing. This has no predictive power or rigor. You're just wishing things would happen as they did before, without considering that conditions and situations change significantly, and instead of Jevon's paradox, we look back at today 50 years from now and talk about Jensen's paradox.

This doesn't mean the concept is BS, but one single concept cannot explain away everything in such a system.

That's when demand is higher than capacity. Now imagine places like Gigalab and Chinese labs are online and able to produce significant percentage of chips. That could cause real surge in prices.

It is incredibly cheap now. What sectors are you thinking of?

There is just so much downward pressure on token price, from every direction. We would need a completely new understanding of economics to explain why the price shouldn’t go down. Or market collusion/regulatory manipulation.

The price has been going down for ages, its not clear what you are pointing at

Pointing at the nay sayers who say tokens are heavily subsidized and it’s all going to come crashing down soon, surely any moment now

I mean, it will obviously crash at some point. With so much pressure on token price to go down that means way less opportunity for margin for AI providers. OpenAI is in a pretty bad situation

What does this have to do with margins? It can remain the same once prices go down

At the price going down? And that it will continue to go down, even if the hardware improvements stop. Not sure what isn’t clear

The demand for them is growing _per person_, not just across the wider economy, if tokens cost half as much but you want to use 3 times as much you're going to have to pay more.

Maybe 1000s of tokens per second unlocks realtime robotic decision making, and now every robot needs to continuously stream tokens to and from the cloud to operate. That could 1000x demand overnight, just to speculate :)

Think about the agents buying computers for their agents. /s

I would very much like it if anything that moves with appreciable mass is governed locally just in case the link drops and/or latency suddenly goes up. Motion is very unforgiving and accidents will happen if that's not taken into account.

Seems unsafe to make locomotive decisions remotely

I think you just found what we will see in the S-1 prospectus of OpenAI

Token prices coming down means nothing if the models keep wasting them

> continue to plummet.

Continue what? The cost per output token has kept going up for the past three years across the board, as thinking models keep leaning more on test-time scaling.

The quality of the said output tokens obviously increased, and arguably increased more than their price, but the price still went up. Or, on the flip side, the price of combined tokens went down (a bit, it did not "plummet" at all though) but so did the average token quality if you count thinking tokens.

With the corollary that old hardware valuations will plummet with them.

Although given we have marginal pricing we need to push through to those lower prices in the face of increasing demand, so timing of this is uncertain and the key to the AI financial markets

this is a story about a proprietary accelerator being built/designed by a token provider. and you think they're going to return the efficiency gains to the customer instead of capture the value for themselves? interesting take.

Yes, I can bet on this happening. If anything, this is a net gain for consumers as it is a competitive market.

go ahead and bet: alibaba is a publicly traded company

We should be mindful of the context that many of these providers VERY likely have been selling their subscriptions at a substantial loss

So as much as i agree “more profits to stakeholders screw the customer”, i think its more of an emergency to get to profitability before the music stops.

> We should be mindful of the context that many of these providers VERY likely have been selling their subscriptions at a substantial loss.

what makes you think this?

Because everyone keeps saying this so it must be true. Real "it is known" kind of vibe with these statements.

He's a subscription truther. There's loads of them. OpenAI's profit increases with each subscription that is cancelled. Pretty soon they'll have more profit than God.

OpenAI just dropped the price of Luna by 80% and Sol by 20-30%

and amazon shipping used to be free without prime, and uber used to be cheaper than taxis, and airbnb used to be cheaper than hotels.

you really don't get it?

almost every pure tech commodity has gone down in price

- gpus

- retail computers

- laptops

- ~gpu~ appliances like washing machines

- cloud computing

i think you don't get how economy usually works in tech

I'm especially enjoying how RAM and SSDs are going down in price.

GPUs and laptops and memory and storage are all crazy expensive

listing gpu's here is crazy considering the current prices

I was checking laptops today for an upgrade from the model I bought back in 2019 and it's not gonna happen from how cheap they are.

GPUs and memory have gone up in price. It's more expensive to buy a 1-2 year old video card than it was at launch, sometimes by a shockingly large factor. Laptop vendors have recently shipped flagship models with less memory than the previous model, because they can't match price expectations for a laptop.

i figured out why this comment is so confusing: this is actually a message from the past, around 2020. either that or simianwords is a time traveler that arrived today and hasn't read the news yet.

Hopefully this also means billionaires can stop trying to drop data centers into residential neighborhoods with zero noise control and polluting on-site generators, signing local politicians on with NDAs, calling for eminent domain to seize homes to build power lines to data centers, etc. etc. etc. Not to mention the water use controversy.

Token prices plummeting is probably a good thing, but not without the regulatory backstops that prevent these effectively industrial facilities from being operated with no regard for the externalities they impose on people who live near them.

Nah, Jevon’s Paradox says that cheaper tokens will mean increased overall energy consumption.

If we can’t even build data centers, the least disruptive industrial use possible, there’s no hope to reindustrialize the US or anywhere outside of China.

We already had plenty of data centers in the US before the AI boom that weren't severely harmful to their neighbors. Cutting red tape is not the same as eliminating meaningful regulation. There are plenty of old industrial sites that could be repurposed as data centers. It turns out it's cheaper to bribe some small town government to give you a tax cut and discounted electricity and water rate.

>polluting on-site generators

how much pollution do you believe modern gas-turbine engines to produce?

>Not to mention the water use controversy.

what percentage of US water usage do you believe is by AI data centers?

Reducing everything to national aggregates provides no insight into the strong negative externalities, imposed on the immediate surrounding communities, of unregulated industrial facilities. That's literally the reason we have zoning laws in the first place,

Counter point: Many AWS services barely decreased their prices (if at all) in the past decade despite advancement in hardware

Yeah but is it really even as good as Rubin? Seems just competitive.

Reply on news.ycombinator.com