And anyone can become a miner. It's not reasonable on small rigs because there are so many miners now, but if most of them leave, then the common man can get back in to mining.
If miners suddenly fled en masse, it’s possible for the chain to be left stranded where the small number of miners remaining couldn’t realistically get to the next 2016-block interval to adjust the difficulty down to match the drastically decreased mining capacity. If mining capacity dropped by a factor of 1000 and it happened right after an adjustment, then bitcoin would be producing about one block every week, and it would take about 40 years (if mining capacity stayed constant) to reach the next adjustment.