The belief itself is probably based on the fact that college grads are extremely likely to have the "other party's" politics. And, that is true because (partly) the higher education system has become quite uniform and (apparently) seeks to squash dissenting opinions among the teaching ranks. How many professors are conservative or openly religious? I suspect those positions are viewed as defacto disqualifications by most on the left.
I recommend everyone read Righteous Mind by Jonathan Haidt. He discusses with nuance and much persuasiveness the importance of returning political representation to higher education. I see it all very differently now.