The costs are so staggering it's becoming clear we're going to be priced out of owning computers altogether. Self-hosting frontier models is a corporation's choice, not an individual's.
You will nothing, and you'll be happy.
It's these 200kUSD+ nodes with a dozen or more data center class GPUs that are killing the self-hosting dream. The cheapest nodes are is in the same price category as literal houses.
Either we get some competition in this space, or computing will go back to their roots as ivory tower big iron.
Long term, HBF shouldn't be more than 4x as expensive as commodity flash, and Kimi K3 is an interesting target for a system using it given how aggressively it compresses the KV-cache. An inference box with ~1.5TB of HBF and ~48GB of DRAM should be able to be built for less than a couple of grand, and get something near to 100tok/s on full Kimi K3 for a single token stream.