I would think quantitative finance is the one area where models are checked, because there's real money on the line. If you're a sell side strategist it might not matter, but if your bonus is on the line, you'll keep an open mind.
I thought the exact same thing going into finance. If real money is on the line, surely they care about which techniques are most rigorously justified in a given model context, right?
Absolutely not. The incentive schemes from clients (who often don't actually fire bad investment managers when they should, and who also engage in chasing returns momentum despite the long and, by now, boring and uncontroversial history of that not paying off) don't actually punish inefficiency. It's a big political mess.
Really the best you can hope for is that they'll hire you for marketing purposes. Hey look, we brought in a shiny new expert in deep learning -- we're cutting edge, we swear! They won't actually let you do any real work with deep learning, of course. It will all be Excel jockey bullshit on factor models in which you'll do obviously fallacious things like directly compare the t-stats of two different model fits as a means for model selection. Maybe you'll write an ineffectual white paper on something slightly more advanced from time to time. But the big reason you're hired is to look good on paper and smoke cigars and drink brandy with the right person who wants you as a political darling in order to win arguments from authority about how you definitely should not migrate away from Excel/VBA.
This is not hyperbole, sadly.
This doesn't count sell side marketing work, or slow money (insurance, mutual fund) buy side work where people get paid for Assets Under Management versus a % of the gains.
Some firms also don't even bother to give any details whatsoever, and the job offer will simply say there is a discretionary bonus, no percentage, no description at all. I actually turned down a hedge fund job because of this. I told them that in order for me to feel comfortable accepting an entirely discretionary bonus, with no baseline or agreed upon way of relating it to base salary or profits and losses, I would need a much higher base salary, and they weren't willing to negotiate about it.
The number of firms, even among extremely quant-heavy hedge funds, who award bonuses in a manner that is not overwhelmingly political is exceedingly tiny.
What this means is that the same political incentives affect even most hedge funds, and so they care far less about the mathematical rigor of what they are doing than about how to sell political stories about it. If "interpretability" sells political stories, then that's what they'll do.
Where are the hedge funds using sophisticated statistics? Maybe Renaissance. Maybe. Where else? Certainly not DE Shaw. Certainly not PDT. Certainly not G-Research. Certainly not Coatue. And on and on. When you interview at these places, and see how the sausage is made, it is eye-opening and alarming to understand just how little their business utilizes or cares about mathematics, statistical rigor, and often not even proper software design. They are just more of the same kinds of shoddy software shops cranking out ad hoc code for rapidly varying political whims, but with far better branding.
I'm trying to get out...
It isn't just about having great models. 51% of the world can't beat the median return. Everyone has great models. It's figuring out where the models are wrong that matter. (Example 1: Most Mortgage models assumed that housing values in all US markets couldn't go negative at the same time) Sometimes that's by qualitative insights. But that's very very hard. And sometimes it's by having someone give some info that they shouldn't.
I think this is expressed in the paper's discussion section:
"We should be careful when giving up predictive power, that the desire for transparency is justified and isn’t simply a concession to institutional biases against new methods."