I know there is some study of it in left-wing political groups, both writing by people within such groups, and some sociological study of them by outsiders. One of the manifestations of dissatisfaction with both Leninism/Stalinism's ultra hierarchical style on the one hand, and the emergence of "union bosses" in the trade-union movement on the other, was a strong ideological attachment to radically non-hierarchical organization among groups like anarchists, council communists, etc. And one of the immediate problems that comes up is non-transparent,
de facto cliques and networks of behind-the-scenes leaders. There's been a bunch of ink spilled on trying to come up with consensus-based organizational methods and best practices that aim to avoid that and produce "actual" non-hierarchy rather than just the superficial look of non-hierarchy. In some brief searching I haven't come up with any particularly good summary to point to, though, and I don't know if there's any consensus (ha ha) on whether any methods are more successful than others.
In a different vein, there's a lot of academic stuff written on Scandinavian workplace management, which is not precisely flat, but has flattish hierarchies based more on rotating and interlocking committees, rather than boss-and-subordinates tree structures. The goal there seems to be to make the sub-groups that are making decisions overt, so informal cliques don't make the decisions. So there are a lot of committees, with rotating membership to make sure they don't develop into hierarchies. E.g. 5 people are chosen to be responsible for something like working conditions, or strategic direction, or integration of foreign workers, for a term of a year or two, and then a different 5 people will hold that power next year. A big controversy there is how much Scandinavian workplace management is tied to Scandinavian culture, i.e. would it work anywhere else?