Hacker News Reader

Full thread

osanseviero

- 180B parameters

- Trained on 3.5 trillion tokens

- 7 million GPU hours

- Quality on par with PaLM 2, outperforming Llama 2 and GPT

-3.5 across benchmarks

- 4-bit and 8-bit show little degradation

View on HN

"4-bit and 8-bit show little degradation" - This is the most interesting part!

It makes sense. Falcon 40B was trained on around 1 trillion tokens. If it was trained to saturation, then Falcon 180B would need to be trained on 180/40=4.5 trillion tokens to saturate it, but it was only trained on 3.5 trillion. And if 1 trillion wasn't enough to saturate the 40B model, then 3.5 trillion definitely won't be enough to saturate the 180B model. If not trained to saturation, then a model can be quantised without losing too much, as it still has "empty space" that can be removed by compression, so to speak.

Reply on news.ycombinator.com