Unless i'm misunderstanding, the only way to get durable collaboration with agents is via the file system. I just mount the subdirectory that contains the source code we are collaborating on, rather than my home directory that contains my .ssh directory, etc.
They only do if you give your container that file system as a volume.