Hacker News Reader

It needs about 22 GB of memory after 4 bit AWQ quantization. So top end consumer cards like Nvidia's 3090 - 5090 or AMD's 7900 XTX will run it.

Just ran this on a 4000RTX with 24gb of vram and it struggles to load, but it’s very fast once the model loads.

A mathematician once told me that this might be because math teaches you to have different representations for a same thing, you then have to manipulate those abstractions and wander through their hierarchy until you find an objective answer.

>I love that emphasizing math learning and coding leads to general reasoning skills

Its only logical.