I discussed that in my comment... But difficulty doesn't adjust fluidly, it's every 2016 blocks. A sudden decrease in hash power causes a sudden increase in block times, which means an increase in the time until the next adjustment
If difficulty decreases substantially, it may convince some to re-enter, but no where near as many as exited; otherwise they’ve wouldnt have exited? Still a huge net loss.
Also, more of a natural equilibrium than checks and balances.