Those are the ratios for memory bandwidth, but the GPUs have a much higher ratio for compute, and that affects prefill rate / TTFT, right?

For local inference, the difference between 25t/s and 70t/s is a lot. For some models I struggle to even reach 15t/s. And "some models" aren't even large models, Gemma 4 13b has this issue for some reason. For stuff like Qwen3.6-27B I can hardly reach 10t/s, even with fully custom inference made by Fable 5!

An old ada Rtx 6000 maybe. A Blackwell RTX Pro 6000 is an order of magnitude faster and has 96gb.