Hacker News Reader

Comment by mometsi | Hacker News Reader

Summary: This is a company that makes AI accelerator ICs. They reimplemented Chinchilla and released the model weights under a permissive license.

View on HN

In other words, they’re actually incentivized to help make LLMs as accessible as possible, rather than try to keep them locked up to hide them from competitors.

Which makes me wonder if Nvidia is doing anything with LLMs too?

NVidia has certainly pushing the envelope on image generation. StyleGAN3 was really cool when it came.

But it is an issue that their chips are hardly optimized for LLMs.

How can a GPU be optimized for StyleGAN but not LLMs? Serious question.

RAM. GPT-3 is over 600GB, ie just the max RAM of 8xA100s, because that's all the hardware can fit.

StableDiffusion plus a whole chain of imagenets can make any visual imagery imaginable in 2GB of RAM. Meanwhile 2GB of RAM barely runs a basic tiny text completion NN that can't do anything intelligent. Text requires a lot more parameters (and more memory/RAM) than images.

The Cerebras node's actual "RAM" (the 40GB of SRAM) is pretty modest too, but being an enormous chip with the networked storage pools is certainly a better situation than a bunch of A100s reaching out to every other A100.

Honestly, all the AI ASIC makers drastically underestimated the RAM requirements of future models. Graphcore's 4GB and Tenstorrent's 8GB per IC is kinda laughable, and it takes them longer to adjust than Nvidia. And Cerebras' original pitch was "fit the entire model into SRAM!"

Have you seen the license on StyleGAN3?

obviously yes, there are larger (20B) models trained on more tokens that are released by NVIDIA under commercial-friendly license on HF

https://huggingface.co/nvidia/nemo-megatron-gpt-20B

Reply on news.ycombinator.com