Einstein's problem was that he used "religion" in a very different sense than what is understood by the term. He was very clear that he didn't believe in literal god(s) or mystical ideas such as reincarnation which are part of non-theistic religions like Buddhism, which are central to the normal idea of religion.