Actually, short selling has another nasty sting in the tail.
If there are few willing sellers out there when everybody goes to cover their short, you get a short squeeze - a feedback loop where the price increases, so people need to buy to cover shorts to avoid losses, which pushes the price up, which means more people need to buy to avoid further losses as the price rises and so on.
The original owner who lent the stock can also ask for it back on relatively short notice (say, because they are long and want to sell). Usually this isn't a problem because the short finds someone else to borrow from and uses that to cover - but if enough people do that then it can trigger a short squeeze even without significant price movement at the beginning, as happened rather dramatically in Porsche shares in 2008.