You're right, but not for the reasons you think. What's keeping xenophobia at bay is not multiculturalism but a reasonably strong economy and social safety net. If these start to fail you will see the xenophobia ramp up as people look for someone to blame for their misery.
I recently visited my family in Alberta where many are struggling due to the crash in oil prices and xenophobia is definitely on the rise. Pretending that "it can't happen" is exactly what allows it to happen.
Do you think that most Germans are, at their core, antisemitic? I don't think they are, but the economic conditions of a 1933 Germany made it easy for many people to blame a specific "other" for their situation with devastating results.