What about teaching non-parametric statistics, like the use of Concentration Inequalities? See https://scottaaronson.blog/?p=3712 and https://en.wikipedia.org/wiki/Concentration_inequality.

Hoeffding's inequality, Chebyshev's inequality, and Chernoff's inequality are broadly applicable, and are therefore less likely to be misused. They also don't require philosophical assumptions about subjective probabilities.

In the theory of Multi-Armed Bandit algorithms, compare the Bayesian approach (Thompson sampling) with the Concentration Inequality approach (the UCB algorithm). The former assumes that payoffs are in the 2-element set {0,1}, while the latter allows payoffs to be in the interval [0,1].

I think Bayesian is cool, but it imposes a big burden with choosing sensible priors.