Because it definitely seems like something which can be trained - professional musicians often have a pain-like reaction to it, while your Average Joe just notices that it sounds bad.
But at the same time, your Average Joe still notices that it sounds bad, and yeah, just like with dissonances, it's hard to imagine someone who doesn't.
Personally I do think that they would dislike it, because I have this theory, which is somewhat similar to / a generalization of the Uncanny Valley theory [0]. That theory is that we fundamentally dislike anything which is not similar enough to something else to be perceived as the same thing, but neither different enough from it to be clearly distinguishable as something different.
So, no matter if that's two tones which are slightly out of tune, a painting with bright blue right next to baby blue, a shadow in the dark, or rice with noodles.