The easiest way to mitigate this is to require two-phase commits on any transaction above X% of the pool.
A flash loan requires that the funds be returned before the end of a transaction. If removing funds is a two-step process, then by definition you can’t use attack it with a flash loan.
Of course there’s always the possibility of someone with that much capital to stake attacking. But the number of participants with $50 million in funds lying around is a lot less than the number with $0. It drastically shrinks the attack surface.