The story in Matthew 19:21 has exactly the same point and precedes this very parable.
> But when the young man heard that saying, he went away sorrowful: for he had great possessions. [Then follows the camel and the needle.]
The rich man fails the test because he is unable to give up his wealth; his heart is not in the right place. He loves the world and he loves his riches more than he loves God. It is entirely possible to have your heart in the right place and be rich, as demonstrated by several other examples.
Another example is in Acts, where Peter kills a man and his wife because they lie about how much money they're giving - they were perfectly free, and would have been saved, even had they withheld their money from the commons. Paul also says there are not many rich men that are Christians - but there are some.
The attitude that "ahah, you didn't give your money to the poor, so you're not a REAL Christian doing what you're supposed to do" is put into the mouth of a figure in the Bible. That figure is Judas.
> Then took Mary a pound of ointment of spikenard, very costly, and anointed the feet of Jesus, and wiped his feet with her hair: and the house was filled with the odour of the ointment.
> 4Then saith one of his disciples, Judas Iscariot, Simon's son, which should betray him,
> 5Why was not this ointment sold for three hundred pence, and given to the poor?
Jesus, and the other apostles, are not saying that you can't be rich. They're saying you cannot love money more than God, and even trusting in money is ultimately a foolish endeavor because your life and prosperity are in God's hands.
If you are snidely arguing that people aren't Christians or following God simply because they haven't given all their money to the poor, you are falling into the same error as Judas, and the same general category of error as the Pharisees.