Hacker News Reader

> It was tested on 4 (A100 80g) and 8 (V100 32g) GPUs, but is able to work with different configurations with ≈200GB of GPU memory in total which divide weight dimensions correctly (e.g. 16, 64, 128).

so we looking at crazy prices just for inference. RIP to the first guy's cloud billing account who makes this public

View on HN

With DeepSpeed an SSD and lots of RAM you should be able to run inference with a 8GB card. There’s a thread somewhere on https://discuss.huggingface.co/ doing the math around this

so err the cheapest A100 i could find was EUR 10.579,79 .

Suddenly that 3090 i wanted to get, does not seem so expensive....

Reply on news.ycombinator.com