Look at Bigtable [1], for example. Tablets are organized in a three level B-tree-like hierarchy, where data is only stored at the lowest level. Is this like what you're talking about?
The downside of sharding is operationally it's a pain, and operational pain translates into pain either in your wallet or for your employees or customers, none of which you want.
Hierarchy is supposed to give you management, access, and performance gains, but if it's too specific you have to use the data exactly how the database wants you to and then you can't take advantage of general interfaces. It shouldn't get in your way, it should just give you free performance boosts with known guarantees and be as simple as possible while also surviving scaling.
I'm not saying this is easy, or even feasible, but it's where we should be headed if we want dev, ops, and the people in suits to be happy.
I am really curious about this premise. Isn't it only a pain if there's no direct system support? (i.e. you have to manually rebalance shards, split up queries at the application layer, etc.) Isn't this the whole thing that NewSQL systems are trying to solve?
Moreover, I don't quite understand what benefits a hierarchical structure provides that a flat (or mostly-flat) partitioning scheme doesn't, ceteris paribus in terms of system support. Especially: doesn't a hierarchical structure mean it takes O(log n) round trips to resolve a lookup rather than just O(1) as in a flat partitioning?