Even direct democracies have restrictions; it's the arbitrary ones that need some real validation to support. Just cause "throw all the bums out" feels good does not mean it's sound policy for a republic.
Whether or not one agrees with it, the concept, at least, of term limits, isn't wholly without merit. It may be feckless, it may be antithetical to the direct democracy many Americans inaccurately assume they live in, but it isn't inherently just some partisan, feel-good policy.
This particular implementation of the concept may be, but that's a separate conversation.