Large communities will literally genocide you and everyone who shows a hint of sharing your belief system, do it through euphemism so he general populace has plausible deniability, burn all the books that refer to you with any sentiment other than poison, and imprison anyone who mentions you in passing while smiling.
With your reasoning, feudalism is the most accountable form of government, safest for minorities.