You have to start somewhere before you can build towards an economy of scale that makes it affordable. If people used your same logic about regular airplanes, we wouldn't be using them today.
It's not more viable now wealth inequality just means that there are more rich people to benefit.
You could have used the exact same argument for normal flight 100 years ago. Supersonic flight being more expensive than normal flight doesn't mean it will always be too expensive for the masses unless you assume there'll be no more economic growth ever.
The low-volume, high-priced rich-person toys (Roadster) finance the scaling up that enables lower-priced mass-manufactured product (Model 3).