It's also quite a bitter taste and coffee-drinkers may have more flexible palates and diverse (generally more healthy) diets as a confounding factor.
Rather than calling their palates more 'diverse' I would just observe that many foods we know to be healthy exhibit bitter, astringent, tart, sour, earthy, etc flavors.
Also this additional stress could elicit some kind of supercompensation by the body via hormesis.