To add an anecdote to this, I have a friend who used to work for an insurer and now works in reinsurance and regularly jokes that everytime an insurer pays somethings, someone is getting fired.
This is ihmo one of if not the most dishonest field of work in the modern world.
There are some companies addressing this problem for consumers.
Look at Lemonade.com, they do not benefit from not paying out as their margin is fixed or Laka.co makes money when claims are paid.
(Disclaimer: I work for Laka)
Lemonade isn’t fixing anything for consumers except maybe UX.
If there is one thing I’ve learned over decades of startup and corporate experience, people that sell on ‘transparency’ are generally full of shit.
Lemonade reinsurers the vast majority of the risk. Their historical profitability will determine what reinsurers charge them (which is a premium). Basically all of their risk (75%) is reinsured.
If/when reinsurers tire of thin margins and high volatility, they'll hit Lemonade with a rate increase. Lemonade can either retain more of the risk, or pay the reinsurance premiums. The additional capital required to do that will either come from policyholders or shareholders.
Events worth insuring against are so rare that you can't use experience from the last time to pick a better insurance company.
Hence, why the market is regulated.
Actually, I think my both renters and auto insurance contains a legal assistance insurance component, which is probably useful if I need to sue the insurance company :)
(I'm not sure it's as large as it ought to be, but the idea is nice)
To some extent, you get what you pay for, also.