The problem is that the side that organizes always wins over the side that does not. And it's very difficult to ban political organization (which is ultimately what parties are) in a way that is actually enforceable.

The American founding fathers were mostly of the opinion that political parties are bad and should be avoided if republic is to stand. Yet they found themselves organizing into parties before the ink was dry.

So the best we can do in practice is engineer the political system such that the damage from party groupthink is minimal.