I'm happy to see someone thinking about latency about like I do. This Microsoft study (https://youtu.be/vOvQCPLkPt4) keeps lingering in the back of my mind.
The most annoying latencies I've seen so far are scrolling on Android and Trackpad mouse moving on Linux (x11).