Hacker News Reader

>Open source in AI means we should be able to reproduce their exact model.

like: Use this specific RNG seed and train for 10 trillion cycles in this data to get X duplicate model?

Practicality of reproducing training runs that cost tens of millions aside, it's hopeless. Determinism is hard enough with a single GPU, fixing a seed isn't going to be much help when training is distributed across hundreds of GPUs.

Reply on news.ycombinator.com