This goes along with another famous teaching of Jesus about money: “No one can serve two masters; for either he will hate the one and love the other, or he will be devoted to one and despise the other. You cannot serve God and wealth.” (Matthew 6:24)
The rich generally have a master they are unwilling to give up. It is money. They think they are the master of their money, but their money is their master.
Yet by God's grace, He can even change the heart of the rich to love Him instead of being devoted to money. Praise God for that. It is indeed a miracle. As crazy as a camel going through the eye of a needle.
We all like to think highly of ourselves, but if we were to find ourselves in that situation, how would we react? Most of us would behave exactly like the rich man. Few have the integrity to meet such austere moral standards. Few of us have the humility to admit that we're no better than the rich man.
And so, in a very real sense, as you also note, there is nothing especially important about the rich man per se. The poor can lust just as much, if not more, for money than the rich, because the poor can more easily fantasize about and project hopes onto what they do not have; the rich man is more likely to have become disillusioned with his riches.