Hacker News Reader

Or, they can figure out something else out? I recall couple years ago when China didn't have enough GPUs (still don't?), DeepSeek team figured out how to train with less computing. IIRC they made Mixture of Experts mainstream and made really optimized kernels and clever use of PTX instruction set.

View on HN

"China does it in a cave with a box of scraps" is a myth. Chinese labs play the shell game to get their hands on a lot of compute outside China.

Tricks like distillation save compute in the RL leg of the process - where a lot of the frontier labs puts their own training run compute.

I'm sure they do, but its not 0 or 1 thing.

Reply on news.ycombinator.com