So where are these training costs getting paid from?
Big AI labs are not software companies where payroll dominates expenses. They're capex-heavy industrial entities; it just so happens that the "machines" (whose output they sell) are nominally the same category as the devices that their knowledge worker employees use on their desks.
Anthropic was profitable last quarter.
If Anthropic can block distillations somehow (which are fair game imo given that Anthropic et al did the same with the written works of mankind), then they might stop or slow down the chinese from catching up.
Chinese also have like 40% of the AI researchers of the world, plus they have access to a lot of cheap labour for writing training data. I'm sure an hour of training data creation from one of China's 162 million university educated people is much cheaper than an hour of work from one of US's 97 million. Probably still cheaper than someone from the grand area.
China is behind in AI chips/GPUs but they are catching up. One thing where they have a hard dependence on outside is their energy imports: they have to import a lot of stuff from third party countries. The US on the other hand is energy self sufficient.
It may be that US labs use Chinese models for distillation but we'd ofc never know because they can host the models themselves
If you feed the most recent Github repos into the training, most of that code will be written by frontier LLMs. Training on that is distillation.
It doesn't make sense to compare OpenAIs or Anthropics compute spend to that of our average software company, because different products require different raw materials. Dropbox also use way more storage than Snapchat, that's an equally silly comparison.