The term 'fief' comes from Medieval land tenure, where it's land granted to a lord in exchange for feudal service. Pop medievalism holds that feudal lords had pretty unlimited rights (the reality is that it's far more complex), and so the term 'fiefdom' in modern contexts carries a connotation that the ruler of the fiefdom is ruling it tyrannically. Another element of the modern connotation is that a fiefdom is usually small, and the ruler is acting as if it's rather more important than it actually is.

In short, it's a backhanded insult.