DeepSeek-V4 should use only 5GB for context due to CSA and HCA, see figure here: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro
But not every framework implements it properly yet.
But not every framework implements it properly yet.
0.07.015.888 I common_memory_breakdown_print: | - Host | 170038 = 162913 + 6740 + 384 |
The 6740 is the cache size.