Most students really dislike anything that appears arbitrary which I think plays into this. One pre-med student I knew despised math after being taught 0!=1 because it seemed seemed completely made up to him and broke established rules.
Part of the onus falls on instructor to clear up areas like this (e.g. explain the cardinality of the set containing just the empty set is 1, 0!=1 allows for many formulas to be expressed tersely, etc). The other component of this is that the students who dislike ambiguity self select out of the program.