Politics being a single bar is one of the problems. People have values that do not fit in any one place on the bar.
And if anything, I think that this classical projected political spectrum isn't a line, it's really a circle. Extreme left and extreme right are the same thing; the particulars of the beliefs don't matter (if you think they do, you can always imagine this being a flat strip of paper glued into a Möbius strip).
It's not just you - this is named "Horseshoe Theory".