Reducing latency isn't about how noticeable it is. Latency can be completely impossible to detect for you but still hurt you.
Input lag is the time between providing some input, such as clicking with your mouse, to getting feedback of this event on the screen. As the clicking will be prompted by things happening on the screen, input lag acts as a command delay to everything that is done. The most interesting feature of latency is that all latency is additive. It doesn't matter how fast or slow each part in the system is, none of them can hide latency for one another. Or, even if the game simulation adds 150ms and your stupid wireless mouse adds 15ms, the 2 ms added by the screen still matter just as much.
The second mental leap is that the human controlling the system can also be considered to be just another part in the pipeline adding latency. Consider a twitch shooter, where two players suddenly appear on each other's screens. Who wins depends on who first detects the other guy, successfully aims at him, and pulls the trigger. In essence, it's a contest between the total latency (simulation/cpu + gpu + screen + person + mouse + simulation) of one player against the other player. Since all top tier human players have latencies really close to one another, even minute differences, 2 ms here or there, produce real detectable effects.