Even if he is a lot poorer than he claims, he is still rich compared to what we normally consider rich vs poor. I'm not sure why we would suddenly change the baseline in this particular case.
If you want to claim he isn't as rich as he claims, then then I'm not disputing that at all (I haven't a clue), but to claim he isn't rich does not align with existing evidence without fiddling with how we define rich and poor in ways we don't otherwise fiddle with such definitions.