Not true, many Semitic Jews who fled from those lands due to persecution went to Europe and North Africa, that includes Ashkenazi and Sephardim Jews.
The difficulty I have with your statement is akin to denying a white, blue eyed aboriginal in Australia their heritage to the land just because one of their ancestors slept with a European colonialist - its fundamentally racist. White skinned blue eyed Australian aboriginals exist.