Hacker News Reader

Yeah, they say it will be "chinchilla-optimal", which means that it probably will be < 70B, might be actually much less as I've seen some recent work that 20B models are able to compete with GPT task [0] so I guess it might be using it but probably isn't, so there's further room for improvement.

[0]https://www.reddit.com/r/MachineLearning/comments/y4tp4b/r_u...

View on HN

What's "chinchilla"? is that a codename for a specific CPU/GPU?

EDIT: no, it's a competitor with GPT-3 [1]

[1] https://analyticsindiamag.com/deepmind-launches-gpt-3-rival-...

More relevantly, it means that the model parameter count, dataset size, and training procedure are balanced such that they can achieve equal or better performance as GPT-3 while being smaller.

Seems like that 20B model is available on HuggingFace [0]

[0] https://huggingface.co/docs/transformers/model_doc/ul2

Reply on news.ycombinator.com