It's pretty easy to follow the evidence in most situations, actually. That doesn't make me an expert. That makes me a person who understands the methodology of science, logic, and statistics. You have to know precisely what your assumptions (observations) are and how to make correct conclusions from those assumptions. The only constraints are your time, your determination, and your ability to get the original research (a constraint is not "guaranteeing your interpretation is correct").
Some people are highly opinionated and don't have the understanding to match. That doesn't mean you can't have any sort of understanding as a non-expert.
"As an aspiring physicist I feel helpless every time I'm at the doctor's office. I cannot just "follow the evidence"."
Doctors aren't necessarily better trained at making inferences and recommendations. Consider their education. Some of them are very poor thinkers.