That's a very wrong way to understand what democracy means. If you're incompetent, you are still afforded a vote. And you sleep on the bed you make.
and in any case, the incompetence voter implies that they could be voting "wrong". This is a deep and insidious implication because implying something can be wrong also implies the other "competent" voters are "right". The very notion that one side is "right" is very anti-democratic.
Is it selfish? Probably, and I'm open and proud about that.
Individualism as a political philosophy would be smth like advocating for libertarianism/adjacent stuff, etc. The apolitical stance is the apolitical stance. I do not engage in politics unless it's a minimum-effort activity like the default mail-in voting (WA). Sure, I may know approximately what the best way to organize society is, and if aliens landed and put me in charge I'd become political; but as it is, I do not care, mostly because it's not productive, but in no small part because actively caring about how to organize society is heavily correlated with being very wrong (IMHO).