Regular Qwen 3.6 benchmarks slightly better and has much wider software support though, so this is probably of interest only to organizations which disallow models trained in China.
2,583 karma · joined August 9, 2023
Regular Qwen 3.6 benchmarks slightly better and has much wider software support though, so this is probably of interest only to organizations which disallow models trained in China.
Not to say the message of the article is completely without merit - there are things to see and do almost everywhere. But if I just get in the car and start driving I will 95% of the time find only strip malls and cornfields. Perhaps a suburban park with some trees.
Kicking myself for not buying the Q6A at the beginning of the year (I wanted three and arace would only sell one per customer, but one would've been better than none).
> can it be slower than without speculative decoding in worst case then?
Yes - running the draft model costs compute and memory bandwidth, and running the drafted futures through the main model costs compute. If the draft model were really inaccurate or you're already compute-limited (usually: running large batches) you would expect some slowdown.In practice, for single-user (non-batched) inference with a working configuration, you pretty much always get some speedup. For non-coding tasks I've seen it be nearly a wash for some people, in which case you might want to avoid it due to the extra memory usage (you'd rather use that memory to run a bigger quant/model, even at a slightly lower speed).
(I would assume they haven't made a lot of $ off of this, if nothing else because they've only just put out that post and demo. They do seem to have produced a model that doesn't sound very LLM-y to my ear, though it also seems rather weak for its size.)
I also refuse to go to Costco these days. Every once in a while my memory fades and I agree to accompany a family member or friend, and am quickly reminded why I should stick to Aldi.
> (50) 'basic tools' means a screwdriver for slotted heads, a screwdriver for cross recess screws, a screwdriver for hexalobular recess heads [Torx], a hexagon socket key, a combination wrench, combination pliers, combination pliers for wire stripping and terminal crimping, half round nose pliers, diagonal cutters, multigrip pliers, locking pliers, a prying lever, tweezers, magnifying glass, a spudger and a pick;
(Excepted devices can require "commercially available tools" which is defined exactly as you'd expect.)
Other than that Zed has a similar experience which is pretty decent.
* By which I mean the good one, whatever it's called now - the part of Copilot that used to be a plugin and is now part of VS Code, not the thing that has always been part of VS Code.