De-federalizing destroys a lot of the political and economic clout that the US can hold over other, smaller countries. This is to the advantage of all the states; I don't think any state seriously wants to secede from union and lose the benefits the federal government entails. No one is forcing the states to "hold the union together" against their will.
Other benefits of a federal government: shared currency and unified economic zone makes for more efficient interstate trade (see: Germany); shared federal agencies make for more efficient shared resources vs every state doing its own thing; etc.
And more specifically, re your statement "I am not sure why people in California have to live under the same rules as someone in Georgia:" they really don't. States write their own laws (in addition to federally-imposed laws for all states). Example of the day, some states have laws legalizing marriage between same-sex couples, other states have laws explicitly forbidding marriage between same-sex couples.