But you cannot actually do that. You cannot be an expert in every field and actually follow the evidence to make up your own conclusion (which btw does not guarantee that your interpretation of the evidence is correct).
As an aspiring physicist I feel helpless every time I'm at the doctor's office. I cannot just "follow the evidence". Heck I cannot "follow the evidence" on anything in physics unrelated to my field.
This is more generally an economics problem about information asymmetry and it will become more prominent as professions become increasingly sophisticated and specialized. I don't think better access to information makes up for it, either, because there is just as much misinformation.
The solution seems to me to build ever more complex social institutions with complex trust relationships which I know makes some cryptofans cringe.