100% - especially when Auth stands for just Authentication.
Simple RBAC authorization also won't take us far.
But Fine-grained Permissions(e.g. OPA, Cedar, OpenFGA, Permit.io) with ReBAC giving ai-agents Zero standing permissions, and only deriving on the fly the least privilege they need / got consent for, can dramatically reduce the problem