I know that stuff happened, but I don't think it's some sort of universal explanation for everything which happens in US politics.
I think the obvious answer here, insofar as that's true, is that historical continuity is simply not as strong as you are assuming. I mean you've basically admitted it yourself. In 1860 would you vote for the Democrats or the Republicans?