Ironically, Jensen Huang did something like this many years ago. In an interview for his alma mater, he tells the story about how he had bet the existence of Nvidia on the successful usage of a new circuit simulation computer from a random startup that allowed Nvidia to complete the design of their chip.
Lumi: https://www.lumi-supercomputer.eu/lumis-full-system-architec...
If you know what your application would be and have the $300 million custom chips may be way more wise. Something you'd only get if you make things in-house/at startups.
Like... how you feel when you use them? (-:
Also:
> For visualization workloads LUMI has 64 Nvidia A40 GPUs.
- low-budget: tax payer supercomputer for tax payer phd students
- high-risk tolerance: tolerate AI cluster arriving 5 years late (Intel and Aurora), lack of AI SW stack, etc.
- High FP64 FLOPs constraint: nobody doing AI cares about FP64
Private companies whose survival depend on very expensive engineers (10x EU phd student salary) quickly generating value from AI in a very competitive market are completely different kind of "AI customers".
Successful startups are successful because they do exactly that. Successfully.
On the other hand if you can't get H100s then nothing to lose!
I can't imagine the GPU would cost more than $100 at scale, unless they have extremely poor yields.
It's mentioned about 2 minutes into this video [1].