Hacker News Reader

Yeah make sure you're using MTP and potentially tensor parallelism.

yeah, i consider that slow.

Oh, ok, that's like the average tps for most AI providers

Whats your setup? I have a single 3090 and am struggling to get it purring

5900x, 3090 24gb (slightly undervolted), 128gb ddr4, running via Ollama.

I am benchmarking it now locally, will put the results and speed/tps on aibenchy.com

How did you undervolt the 3090?

Msi afterburner, you go around 900mv curve editor, raise it up to normal clock frequency, and it uses like 260w instead 300w for same performance.

There's some YouTube guides for it.

I also undervolted my new 5070ti, same tdp, around 260w instead of 300w and like 8% better performance.

I remember the days of crypto mining memory intensive tasks do not suffer from TDP reduction as much as game rendering.It's so counterintuitive.I remember getting 96% performance at less than half of TDP with my old AMD card.