I do agree with your final point-- if social media went dark tomorrow, no ones rights would be diminished. But if it went dark for only certain people, I think we would agree that -something- is being diminished (even if it's not necessarily a right or that it's in the best interest of everyone).
I fully believe that this is a topic where people get to land differently, and I respect those that do their mental calculus differently. There's so many second-order and third-order effects when it comes to speech, and then you amplify it to global-level... there's no great, clean answer. But ultimately, we get to choose what we weigh as most important-- as I've heard others on HN say, "If we wrap ourselves around every conceivable axle then nothing will be achieved."