Hacker News Reader

What will be the approx token/s prompt processing and generation speed with this setup on RTX 4090?

I also just made IQ1_M which needs 160GB! If you have 160-24 = 136 ish of RAM as well, then you should get 3 tokens to 5 ish per second.

If you don't have enough RAM, then < 1 token / s