The Economics of Open-Weight Inference
data.ornn.com
data.ornn.com
This is clearly not true, and we are starting to see compute markets, but only for rental prices/H, not for the hardware itself. I feel like there is some artificial moat being built here to stimulate sales of new hardware, because an H100 at 1/16th the price will have comparable dollar/FLOP as Vera Rubin.
Which banks have analysts that understand the difference between H100 and A100? Do you have actual experience with being denied a loan based on this or are you just making things up?
The claims of many GW of installed training/inference have come under scrutiny lately. The first VeraRubins aren't even there yet, so it's all GB300 NVL72s as the peak performers and probably <<1GW of those so far. Even xAI Colossus is mostly H200s and B200s.
Electricity costs are also a huge differentiator. When drawing 100kW the difference between >50cents and <10cents per kWh is pretty big! One is almost $0.5M and the other is less than $100k.
I've taken to using them as micro-review subagents at development milestones, where a "frontier - 1" model like Opus or Sol launches 10-15 of them on small review tasks that each run for ~10 minutes. Costs about $1 per cycle, and they usually catch something Astra or Fable didn't. Then the orchestrator validates each claim before passing it back to the planning session so we can fold the findings in.
Perhaps because they are dumber, they produce better results? IMO an excellent well-tuned harness combined with a "frontier minus x" model produces the highest quality result. DS4.1 and Qwen3.8, far from being a compromise, legit give me better results. For my personal definition of "better".
And once frontier models can architect (turn business requirements into engineered systems) then I guess no one needs a job because that’s the digital singularity.
Perhaps they simply don’t advertise it and investors (currently) love companies that spend heavily on frontier models. That said, OW models, especially when combined with RAG, work quite well, and given the current state of the industry, they may be the only sensible way to keep inference costs under control.
Why do you think OAI etc are all up in arms? They hate what’s going on. Most here are delusional.
I work in a very large market cap firm and i’m telling you - more and more managers are pushed to get their teams to use open source and squeeze employees to get the max out of them.
But I'd still rather use them since it's inevitable the unsustainable margins of the closed weight providers lead to enshittification once they stop subsidizing demand
I'd rather spend more today with a workflow whose underlying unit economics are sustainable and don't force me to inevitably look for other solutions once the closed weight providers start optimizing for profit and extracting value
It also further exposes the “we’re profitable this quarter” claims coming from big labs. These statements increasingly look like WeWork “community adjusted EBITDA” nonsense of “we’re profitable if you ignore all the reasons why we’re deeply unprofitable.”
The point about no salvage value: the paper is really pricing a rental service, not the devices, and even there the 5-year A100 term holds 80% of the one-month mark. The reason there's no clean resale market yet isn't that old GPUs stop earning. It's that the workloads keeping them earning (batch, RL rollouts, long-horizon agents) are the ones that tolerate latency, and the people running those tend to rent rather than buy secondhand.
The part the paper understates: rented A100s aren't the biggest pool of already-paid-for, latency-tolerant capacity. That would be the Apple Silicon and flagship phones sitting idle, where the capex is fully sunk and there's no hourly meter at all. Same economics, one step further. The deciding factor is who gets to make the deployment decision, and open weights are what move that decision to the operator.