Hah. I did pretty much all the same stuff in my previous company.

One thing that we did a bit better: we used AWS SSM to provision our SSH-CA certificates onto the running AWS EC2 instances during the first connection.

It would be even better if AWS allowed to use SSH CA certs as keys, but alas...