It's practically an axiom.
How do you turn a good idea into an existential threat to humanity? Let a like-minded group of very rich people commandeer it for a short while.
- their evidence is true (eg it's not just a too small sample size, in very particular conditions, etc)
- and MORE IMPORTANTLY that they haven't forgotten OTHER EVIDENCE which points to a much better way to do it
usually when you "see the world through metrics" you start to be very efficient at optimizing them... but then the first minute your metrics start pointing a different direction to the one it's "supposed" to (which happens to 100% of metrics since "all models are wrong but some are useful") and you go with the full speed of your optimization on the WRONG target, you start getting into issues quickly...