32GB dedicated to an N-gram table instead of a draft model is an unusual choice for speculative decoding — what made it win over the more common draft-model approach here?
It wasn't either/or, the N-gram table is part of Qwen itself and stays on disk. I’ve now added its 1.5GB MTP draft head too, it gets 86% acceptance and about 1.24× faster decoding on my 48GB Mac.
Got it, thanks for clearing that up. 86% acceptance is solid — does it stay flat over longer generations, or drift with context length?