Perhaps we can blame exposure to actual pathogens which resemble the pollen in some way, triggering a misdirected immune system response.