That's great feedback. I'll try out this specialization on a simple struct (int32/string) combo later today. We can still get rid of the indirections around slice manipulation (Swap and Len) but obviously stay stuck with the Less.
Or try sorting points (struct of two float64s) by the x-coordinate, and compare the timing with similar JavaScript code running in V8. You might be surprised...