You would have a big group of people who would really struggle if they ever had to play something familiar transposed, rather than just a handful.

What the post you're replying to is alluding to is all the ways that pitch is simply not absolute. That can be for boring reasons: you're playing with an organ, or some other instrument with fixed or temperature dependent pitch, or in a different city with different conventions; but also for the interesting reason that the "correct" pitch of a C, say, is the result of a mostly-unconscious negotiation that considers ones' place within a chord that's sounding and the melody you're playing. Even equal-tempered tuned keyboard instruments involves compromise wrt placement of the octaves, etc.