Just saying you don't like a form of government makes it illegitimate is a silly proposition. You get people every election in democratic countries saying the government is illegitimate. Usually they're considered crazy. Should "tyranny of the majority", as they'd say, be considered legitimate? What about tyranny of the minority, since in many countries, most people don't even vote?
You can recognize that a dictator is the legitimate ruler of a country, while also acknowledging that he's not working in favor of the people. If someone has an entire territory's military at their command and they can write and enforce laws, that's usually a sign that they have a legitimate government.