Hacker News Reader

10 billion. The number of FLOPs has gone from ~1e+14 (6 layer MLP MNIST) to ~1e24 (Megatron-turing NLG).

edit: s/parameters/FLOPs

Not parameters, the amount of FLOPS required to train the model.

Whoops, thanks! That's what I get for typing too quickly between household duties.