Claude is a mystery/surprise to me. My mental model has been to train these cutting edge closed source models you need 1) Bespoke supercomputer (no public cloud will cut it) 2) Great dataset (which takes a long time to collect unless you have a partnership with with a search engine) 3) Couple hundred lines of pytorch code to run on the supercomputer 4) A couple of employees with experience in the dark arts of malfunctioning GPU's and exploding gradients

Anthropic is a relatively new startup that probably has 3) & 4) from their history at OpenAI. But I don't see how they could have 1) & 2).