Open Router
Input /M $0.45
Output /M $3.20
Cache read /M $0.05
Throughput 27 tps
It would be a very nice model at 200-300 tps and if it was dirt cheap. What's the limiting factor of optimizing speed and price for inference providers?
Also, DeepSeek's KV cache requires less RAM than Qwen's. In concurrent situations (on servers) you load model weights once, but you have different context in each parallel session. So, it can also need less RAM than Qwen to serve, even if it's a larger model.
More like 3-6.
Qwen 27B full quality is FP16. So 54GB. In practice most inference providers would serve FP8, so 27GB.
DeepSeek V4 Flash in full quality is mostly FP4. ~167GB official release.
So Deepseek has 140GB model size overhead... which is shared between 100s of users single inference node serves, so not even a gigabyte of VRAM per user.
Memory required for 200K of context per user:
V4 Flash: 1GB.
Qwen 27B: 13GB.