I see a world soon where there’s an extremely wide variety of small models for speculative decoding, unique to use cases, companies, and even individuals.
this is definitely where things are going. the enormous "eat the world" models have extreme diminishing returns by comparison.