I agree mathematically but wealth is very much a relative phenomenon, hence the term "inequality". Yes if an alien dropped a bunch of real wealth on the planet and gave it to a lucky individual, the others would be no poorer, but they would sure feel it (all else equal).

This is obviously a highly complex and lengthy topic that spurs tons of thought experiments but I generally agree with your latter point.