If you use 1000hz polling like my latency tester and keyboard do the latency becomes less than 1ms which is pretty small compared to the other factors. See https://michael.stapelberg.ch/posts/2018-04-17-kinx-latency-...

I'd optimize lots of other things like trying to avoid the compositor, higher refresh rate monitors and lower travel keys before I'd look at USB3.