The matter is that he is specifically denouncing Jews in their link with the USSR, that's what makes him worthy of being called antisemite. It's not merely a coincidence (i.e. he denounces a specific social group which happens to contain many Jews), he singled them out in the context of denouncing the Soviet Union.