This one works great. https://github.com/ormandj/sglang-deepseek-v4-flash-sm120
172 karma · joined November 18, 2019
In other words, i would like to explore a way to approach a problem of statically scheduling (heterogenous) computations and data transfers between multuple tiers of caches (disk, ram, vram) in OR fashion (e.g. formulating MILP).
In other words, i would like to explore a way to approach a problem of statically scheduling (heterogenous) computations and data transfers between multuple tiers of caches (disk, ram, vram) in OR fashion (e.g. formulating MILP).
[0] https://github.com/facebookresearch/faiss/pull/2521 [1] https://github.com/rapidsai/raft