It seems clear to me that most people (even the most brilliant engineers apparently) still need something as a reference, they can't just be let on their own.
But that reference doesn't need to be a single manager. We could start to think of the team as the manager. The team can perfectly give feedback, handle friction, distribute boring work and so on. We already have plenty of stupid daily standups, team meetings, weekly all-hnds meetings and so on, so it wouldn't really be hard to have a team daily standup where problems can be brought up to everyone and solved cooperatively.