> You're "pulling priors out of thin air" whether you realize it or not

No, you are not. That’s an argument I often seen put forward by people who want the Bayesian approach to be the one true approach. There are no prior whatsoever involved in a frequentist analysis.

People who say that generally refer to MLE being somewhat equivalent to MAP estimation with a uniform prior in the region. That’s true but that’s the usual mistake I’m complaining about of reducing estimators to MLE.

The assertion in itself doesn’t make sense.

> Of course, Bayesian statistics also often involves assigning "uninformative" priors out of pure convenience

That’s very hand wavy. The issue is that priors have a significant impact on posteriors, one which is often deeply misunderstood by casual statisticians.