Action: X publishes something properly critical of the right/left.
Reaction: The extreme right/left media throws all sorts of criticisms of X's reporting and sees what sticks.
Result: Some fraction of the right/left constituency starts to believe (and voice) that X is biased.
Action: X publishes something wrongly (or at least with more bias than expected) of the right/left
Reaction: All right/left media publishes articles lambasting X's coverage of the item in question.
Result: A fraction of the more moderate right/left constituency starts to think "maybe my extreme friend was right about X"
Cycle through the above a few times, and even though the majority of the country may trust X, most of that majority will be among the least politically engaged (because partisanship and political engagement are correlated), so you are left without much of an audience that cares about what you are reporting.