Could you elaborate a bit more how you get the containers into their own IAM roles?
Queues map pipelines to agents. Agents can be assigned IAM roles. If you want a certain build to run as an IAM role, you give it a queue where the agents have that role. For AWS, Buildkite has as a Cloud Formation stack that sets up auto scaling groups and some other resources for your agents to run.
Basically it adds a signed web identity file into the container which can be used to assume roles.