Depending on your requirements, yes that's kind of what happens if you want to centralise. It can make sense for Google-scale problems where you really do need to handle the complex graph of relationships between all users and resources, and doing that in any one service is non-trivial.
In practice though, a lot of service-oriented architectures can get the same benefits by having a central user management service, and keeping most of the authorization in each service. That central service can provide information like what organizations/teams/roles etc. the user belongs to, and then the individual services can make decisions based on that data.
This is the approach I covered with the hybrid approach. With this you can still implement most complex authorization models.