What many libertarians want is decentralization. As much as possible. They probably want to get rid of all powerful Federal Government but local governments (as small as possible) and confederations are usually fine.
Many people seem to be complaining about certain absurd EU regulations that were forced upon us.
I, for one, am extremely happy - centralised government has saved us from a lot of absurd ideas the local policymakers had. Also, we have one extra level of courts, where you can go when your government breaks law.
Not to mention all the benefits for commerce and travel - knowing that other EU states have similar laws and standards helps a lot.
I find state rights advocates to be the worst™ people in the world. They advocate for state rights when it suits them. Absolutely hypocritical bunch.
We've decided as a nation that there are some rights and responsibilities that we have regardless of where we live within this nation. How many times do we have to fight for civil rights?
Besides, locality has inherent advantages such as familiarity, cultural similarity and accountability. So at least some local governments will be probably suitable for your liking.
The power of two or three wealthy families is readily visible almost like a movie in a small town. The power structure in a bigger city from what I can gather is a little bit more complex since there are more players. However, I don't think "the powers that be" in a local level are any more accountable to the governed in a localized setting than in a national setting.
I mean think about the last HOA you lived in and tell me you felt the power structure there was "fair".
The state rights advocates like I said are the worst. States are too big. What we should focus on is the individual.