It's the ultimate victim blaming religion - you're poor because you were a bad person in a previous life.
Very awkward to jokingly say "I wish I were a puppy" when I play with my dog and to see the look of abject horror on my Buddhist partners face. I'm told even wishing for something like that significantly increases my chances of becoming a dog in the next life - which is permanently an undesirable fate.
You don't think Calvinism is worse? There, you're poor as an indicator that you're destined to go to Hell.
As a free person, I can choose my religion and blame God for a change. The difference is that if you place locus of control further to yourself, that gives you (and not an external entity) enormous power. Yes, you might not be able to change what happened, but you can influence what is going to happen, even if it's not immediately obvious and if some links of the casual chain are hidden.
A slight misunderstanding, or an inconsiderate person may use it for victim blaming, but that's not what lays down. It gives guidelines that our actions have consequences, good or bad even if they are not immediately visible to us. On HN, people talk about the role luck plays in the success of startups. Two different people working hard, with brilliant ideas: one of them gets to build a billion $ company, while the other may go bankrupt. What we call luck, or randomness here is explained as karma (good, or bad deeds accumulated from earlier lives). Used with right understanding, and compassion, this provides good guidance for those who seek it.
If you have positive karma in this life you will be rewarded in the next but the manner of the reward is more or less beyond your control
And that's assuming you believe in that sort of thing to begin with