Could be! I imagine I also notice the difference because I usually use UHT when I've run out of fresh milk. So the taste of coffee with fresh milk is pretty recent, vs travelling to another country.

I'm sure if I was using UHT every day I'd start to forget the difference.