The fear of Turks in Europe solidified after the defeat in Vienna and left a deep mark that predisposed the idea of Europe as a Christian club. Go look around statues and stuff in Vienna to see what I mean. I have Turkish friends in Germany who, given the treatment, experience being called a Turk as a racial slur. In some sense it's good that so many real arabs are in Germany right now, which should show the broader population the difference between Anatolians and Middle-Easterners. The Turkish population is of quite a mixed ethnic background, even today. This is normal for a place that's not tiny and partly due to cross-pollination too.
The way the EU has evolved is highly questionable and the memberships of many are even more dubious to say the least, so there are much deeper problems than whether another place like the UK, just with almost no Christianity, should get laxer trade agreements or not. Think about why some very central European Christian countries don't use the EURO or in the case of Iceland aren't even fully in the EU but could have been.