Correction, the vast majority of users will not be able to explain that they feel a responsiveness issue, but they do indeed feel it. Less hardcore gamers will often just blame other things -- loose controls, or ignored button presses, or what not.

I see it pretty regularly in playtests and such. Anyway, it's not something someone can simply say "Hey, this app took longer than 50ms to respond!" It's more of a gut feeling that something about it is slow.

They are likely aiming at the 50ms threshold because that's the point at which you really can't notice it, most of the time. At > 50ms, it becomes more and more noticeable, and by just 100ms people actively start complaining about the slowness of response.

edit:

As for the FPS gaming issue, the reality is that you don't notice it in FPS gaming because the games are built to hide it from you -- when you press your fire button, your gun fires, regardless of net latency. And that has been true for a very long time. I think the last game to really wait for server response before doing anything was Quake 3, and probably some of the Quake 3 engine games.

I could never stand Quake 3 for that reason, if you had anything higher than 30-40ms it started feeling like you weren't controlling the game.