Key difference is that is one relay author becomes "oligarchical" the notes just route around that (through different relays).
That said - maybe (total hypothetical) the reason one relay becomes really big is because a lot of people think it provides really good service, and maybe it's difficult to convince the majority of the network to route around it. This would create a similar problem to what we see in more well established federated chat networks.