It's double edges all the way around. "shared background" has certainly been the cause of a lot of harm. It's one of those things that people want, but which is bad, like "Everything be more efficient if everyone would stop wasting time on their own different ideas and just did what I want.". There are no single simple correct answers.