If the point is that it'd be difficult and absurd to try and equalize all
outcomes, that's reasonable enough.
If the point is that any attempt to equalize an opportunities is conceptually the same, though, that's much less reasonable.
Education is often one specific focus because it's in the overlap of (1) a common and early place where inequality of opportunity presents itself (2) early maximization of individual potential has potentially cascading effects (3) baseline tractable social projects (raising the ceiling on outcomes for everyone may be hard, but providing a reasonable floor more most is one thing it already does) (4) seems plausible that everyone benefits the more educated the population is (especially in a representative democracy).
And while eliminating fancy restaurants may be absurd, if you're thinking about the question of when/why it might be important for poor people to meet rich people, then it may well be entirely worth trying to address that problem. To pick one issue near and dear to the heart of HN folks, access to capital is key for many entrepreneurial folks.