When we look at hashes mentioned in this discussion where people are trying to be fast, blake3 is at 3000MB/s when vectorized, and cityhash, murmurhash, and siphash range from 5x faster to 2x slower if I take the smhasher numbers. There's not that much difference.
But also the number we need here is the latency of a small hash, not the max throughput. If blake3 takes 250 cycles for that, how much faster do you think you can go? You can't fit a reasonable hash into 10 cycles. You can't even fit o1hash into 10 cycles. And all of those numbers are pretty fast for most purposes.
AES-NI is a great tool! But isn't that arguing in favor of a crypto hash?