Where things usually get tricky is for the CI/CD pipelines and/or the administrative users that need way more access. It is very hard to scope to true least privilege - including for things like lots of ECS/Fargate where you don't want people to mess with each other's Task Definitions/Tasks/Services hosted within the same AWS account for one example. The various AWS services are very hit-and-miss for how well you can scope resources and what conditions they offer you.
Security will say "no resource star" which is a best practice but quite difficult to get right in most larger accounts. Permission Boundaries help in being able to flip the conversation to "lets list all the things these users/pipelines shouldn't be able to do instead of what they should to then constrain the more wildcard-y/star-y IAM policy we need." But even those are getting harder these days because there is soo much you don't want your average operator/admin to be able to do too.
Usually people throw up their hands and over-provision generally or give each team their own AWS account and overly permissive access within it - but ring-fenced to their own stuff at least as a risk trade-off. Though I think the pendulum may be swinging back from a bazillion AWS accounts (with all those problems) back to trying to solve the IAM problem with additional new tools (CIEMs etc) that will help them to manage IAM as-a-service with a pretty UI or by letting you scope down users/roles to only the activity they have done within the last 7 days etc.
There is a great line that "complexity isn't created or destroyed - it is just made somebody else's problem" - do you want to make these an AWS IAM problem or an AWS account-management problem? A pipeline/automation problem or a heavily-staffed security team who can write great IAM policies/PBs/SCPs problem? A SaaS vendor we can procure a CIEM from problem? etc.
We created a policy system that allows us to define these individual minimized policies based upon specific services that we've created. We have a tool that can then combine these small bite sized policies into a larger policy while combining compatible actions and resources giving you a resultant policy that is equivalent but often much smaller than the logical combination of all the individual policies.
You can use this the resulting policy in a variety of ways. It's very easy to just make a custom role, set this as an inline policy, and then use some custom tools to keep the policy updated.
In some cases, we went with a "policy.d" directory in a project source tree that contains symlinks to all the small specific policies it's using, and some deployment commands that use these symlinks to create a resultant policy document. If you want to add or remove a policy to a project, it's as easy as adding or removing a symlink. Likewise, it makes it much easier to audit which policies are actually attached to the project.
In terms of our situation, we provide fine grained access to distributed resources, mainly data elements: think records/fields. An example is to define which user, group, and role can access which records and which fields within each record and to what extent (e.g., can't access SSN at all, can only get last 4 digits of phone number, can see first/last name, etc.).
I really liked the policy approach of IAM so my plan was to let data owners define policies that are then applied to users, groups, and roles. At run time our coordinator engine will check levels of access to each query (that could be to one data store like Postgres or Salesforce or a federated query spanning multiple data stores). By assigning a set of policies (with IAM's effect/action/resource/condition model), we can make this happen in a flexible way as I see it. Effect also has "deny," so that would be very useful for a majority of situations.
A hierarchical model like Google's as mentioned in the article doesn't seem as flexible as this IMO.
In your case, your resource could be a "record" for more global yes/no decisions, or perhaps as a "field" for more granular cases. Things like "can only get last 4 digits of phone number" could be achieved through attribute-based conditions set within the policies.
> I really liked the policy approach of IAM so my plan was to let data owners define policies that are then applied to users, groups, and roles
An advantage of Cerbos is that policies are defined and deployed separately from business logic in (yaml/json) config files, so no changes are required in code when policies need updating.
> At run time our coordinator engine will check levels of access to each query
Can't wrap my head around this particular part - is this checking if an entity can or cannot run a particular query, or specifically based on the "things" the query is returning?
(as a disclaimer I should mention that I work there, although Cerbos itself is Apache 2 licensed and completely free to use)
For some scenarios, resource-based policies can form the foundation of your auth flow. If you look at the flows described in the docs [1], resource evalutionn is simpler. You still need to solve the problem of effectively managing all of those resource policies and limitations on where they can be applied [2]. That might be an easier problem to solve then dealing with trying to express everything as an identity policy. You're then less concerned with wider permissions at the IAM level and move the responsibility to the owner of the resource.
[1] https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_p... [2] https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_a...
As long as you have a constant number of stacks consisting of ec2 (even with individual resources autoscaling), lambdas, whatever really, you can write an IAM policy for that. It might be tricky but generally doable.
As soon you get into a random number of stacks you also get into dynamic IAM generation and that is really hard. Add IAM adjustment for used-based inputs and sprinkle with cross-account access and there you have it: an endless stream of new IAM headaches.
e.g.
* I just want to do something simple => Solution space presented is too complex
* I have a complex use case => Solution space is easy to map my problem into