> It's not that controversial.
It's certainly controversial if used as a defense for why a party 'couldn't win,' which is how I read the original comment. It's not like there is some magical system that conspired to elect the opposite party for Clinton, Bush Jr., Obama, etc. but didn't act for Reagan, Coolidge, etc. The fact of the matter is that poor presidencies (or presidencies that the public did not correctly identify as poor) played a large part in re-election of the party, perhaps more so than the running candidates. And on average, we have had a lot of presidents who leave much to be desired, so it flip flops as a result.
If it's not being used as a defense, then it's just an incorrect point, because as you said, there is nothing statistically significant either way, which is basically what the original comment asserts by declaring it impossible.