I once had a colleague (who is a nice guy, and we got along) who said during lunch that there are some things that we all can agree on, like how rent control is a good thing. I offered one objection off the top of my head (that prices serve as a mechanism for aggregating data, and any price controls serve to throw out data, and we should use as much data as possible when making decisions about how to allocate scarce resources). Before I spoke up, this colleague couldn't imagine that there would be any counter arguments to his position. Another colleague later thanked me for speaking up and sharing that different take.
Maybe that's not a good reason to not have rent control, or maybe it is a good reason, but on balance it's not good enough. The problem isn't the debate, the push back, it's Graham's point "privilege makes you blind." The problem is when there are limits to your imagination that an issue you see as simple might be more complex or that the person you're sharing a meal with might have a different, but still legitimate, way of seeing the world.