Or when you have to pass by homeless tent cities in your downtown. Or when you are the victim of crime because the kid pulling a gun on you is envious because their family is poor and you're rich.
Extreme inequality brings social strife in multiple layers, if you're individualistic you might not care for it but for a society it fosters resentment and erosion of social cohesion.
But I don't think you care about that in general.