Unboxing is a huge optimisation though isn’t it? No need to allocate when inserting a new item into the queue (beyond the standard allocations which happen when growing a slice), fewer pointer indirections when reading the queue, easier garbage clean up at the end (a single deallocation of a slice).
I think (but am not sure) that the Go compiler can add some small primitive types and structs (precisely: those that fit in a machine word) to a []interface{} by storing them inline, removing the need for an allocation. So perhaps you will only see major speed increases when using larger structs.