HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by mromanuk | Hacker News Reader
Parent
Full thread
mromanuk
·
one 4090 + 3090-Ti
https://github.com/turboderp/exllama
View on HN
trifurcate
·
That's not a minimum, just what the author had on hand. I can get 65B and 70B running just fine with exllama on 2 3090s.
Reply on news.ycombinator.com