Its a game of diminishing returns.

Assume there are 80% included in a game and 20% frozen out. Here it makes sense to change the rules of the game to include the other 20%.

Yes it might slightly inconvenience the 80% but the switch from not being included at all to playing the game for the 20% is huge. So, overall it might be a net positive.

Problem is when you get to a 98/2 split. Is it worth it to inconvenience such a large majority for the benefit of a very small minority. And how much are you going to change the rules?

Is it worth the reduced enjoyment/efficiency of the game?

At some point the trade off will become too much, and even moral/emotional arguments will not be enough to silence the dissent.