why do we have guides and lessons on how to use a chainsaw when we can hack the tree with an axe?
277 karma · joined April 11, 2025
I definitely trust the totally private dataset more.
He defines it as "everything that happens from when you put a prompt in to generate an output" -> but he seems to conflate inference with a query. Putting in input to generate the next single token is inference. A query or response just means the LLM repeats this until the stop token is emitted. (Happy to be corrected here)
The cost of inference per token is going down - the cost per query goes up because models consume more tokens, which was my point.
Either way, charging consumers per token pretty much guarantees that serving models is profitable (each of Anthropic's prior models turn a profit). The consumer-friendly flat 20$ subscription is not sustainable in the long run.
https://epoch.ai/data-insights/llm-inference-price-trends
https://www.snellman.net/blog/archive/2025-06-02-llms-are-ch...
Fundamentally, serving a model via API is profitable (re: Dario, OpenAI), and inference costs come down drastically over time.
The main expense comes twofold: 1. The cost of train a new model is extremely expensive. GPUs / yolo runs / data
2. Newer models tend to churn through more tokens and be more expensive to serve in the beginning before optimizations are made.
(not including payrolls)
OpenAI and Anthropic can become money printers once they downgrade the Free tiers, add ads or other attention monetizing methods, and rely on a usage model once people and businesses become more and more integrated with LLMs, which are undoubtedly useful.
0(?): there’s no provided definition of what a ‘world model’ is. Is it playing chess? Is it remembering facts like how computers use math to blend Colors? If so, then ChatGPT: https://chatgpt.com/s/t_6898fe6178b88191a138fba8824c1a2c has a world model right?
1. The author seems to conflate context windows with failing to model the world in the chess example. I challenge them to ask a SOTA model with an image of a chess board or notation and ask it about the position. It might not give you GM level analysis but it definitely has a model of what’s going on.
2. Without explaining which LLM they used or sharing the chats these examples are just not valuable. The larger and better the model, the better its internal representation of the world.
You can try it yourself. Come up with some question involving interacting with the world and / or physics and ask GPT-5 Thinking. It’s got a pretty good understanding of how things work!
> when a new model is released as the SOTA, 99% of the demand immediately shifts over to it
99% is in the wrong ballpark. Lots of users use Sonnet 4 over Opus 4, despite Opus being 'more' SOTA. Lots of users use 4o over o3 or Gemini over Claude. In fact it's never been a closer race on who is the 'best': https://openrouter.ai/rankings
>switch from opus ($75/m tokens) to sonnet ($15/m) when things get heavy. optimize with haiku for reading. like aws autoscaling, but for brains.
they almost certainly built this behavior directly into the model weights
???
Overall the article seems to argue that companies are running into issues with usage-based pricing due to consumers not accepting or being used to usage based pricing and it's difficult to be the first person to crack and switch to usage based.
I don't think it's as big of an issue as the author makes it out to be. We've seen this play out before in cloud hosting.
- Lots of consumers are OK with a flat fee per month and using an inferior model. 4o is objectively inferior to o3 but millions of people use it (or don't know any better). The free ChatGPT is even worse than 4o and the vast majority of chatgpt visitors use it!
- Heavy users or businesses consume via API and usage based pricing (see cloud). This is almost certainly profitable.
- Fundamentally most of these startups are B2B, not B2C
https://chatgpt.com/share/684e02de-03f0-800a-bfd6-cbf9341f71...