I originally was noodling with this in the context of a space civilization, where there is no equivalent of "solid ground", everything's always moving, and everything can move (i.e., even a "space station" can still move places, it just may be slower than a "space ship"), where these sorts of structures are probably inevitable because at the unit of 'space station' you can't hardly stop them from moving around between what various borders there may be.
But you can still noodle with the idea on planet Earth, too. Obviously, our current systems have a lot of institutional inertia in the direction of centralization; for instance, the various impacts this will have on the United States due to how we choose representatives, etc. But you can imagine a world in which this gradually becomes more popular, and imagine what the follow-on effects may be. And also consider why it not how things work here. There have also been times and places in history where things did work more like this, such as when and where "city states" were the dominant organizations rather than "nations".
(I present this as an interesting thought topic, not as the solution to the world's problems. I think one thing you will rapidly notice if you put serious thought into it is that especially on Earth, the nature of militaries has a lot of impact on why we find it advantageous to bundle into larger collectives than our neighbors. But I do find myself wondering whether over the next century or so we could save ourselves quite a few shooting wars if we were all more willing to break up amicably instead of fighting to the death over who gets to control tho overly-large, bloated collectives. Alas... we almost certainly won't.)
(Another example... in the space case, your choice of governance might actually manifest as concretely as the operating system your space ship or space station runs on, with its corresponding grants of permission or lack thereof to whatever higher powers your choice of governance has. And then that raises a whole bunch of further interesting possibilities.... on Earth though I would expect geography to still loom large on all discussions of sovereignty, though.)