I think that's not right for two reasons. One is that legitimacy isn't equivalent to democracy. I think it's more appropriate to define a legitimate authority as one that acts in the interests of its constituency, democracy is just a process for bringing authority into power. There are non-democratic institutions or governments around that are legitimate, and there are democratically empowered governments that are illegitimate. Extreme cases including religious or ethnic populist movements that end up slaughtering substantial parts of the population.

Second reason is, I don't think most non-democratic governments hold power due to application of violence (few exceptions, mostly small). If the people of China decided to march on Beijing tomorrow nothing in the world could stop them, they've done so in the past. The reason they don't is that they see the government as legitimate, even if not democratic. Basically the same way corporations work. Very few are democratic, but they're not really upheld by physical force. The Catholic Church still has non-democratic legitimacy over a billion people. Clearly that's not because of the 500 Swiss Guard members in the Vatican.