HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by test6554 | Hacker News Reader
Full thread
test6554
·
So a single RTX 4080 ain't gonna cut it. :-(
View on HN
lodovic
·
This is about the price of 3 5090's, but it can run the full 670B model with Q8 quantitization, albeit at a few tokens per second. Doing this on GPUs would require 24 5090's (if nvlink still works, but I don't think so) or 10 H100's.
Reply on news.ycombinator.com