Using the SSM tool as a ProxyCommand for OpenSSH is brilliant. I can SSH to an instance (turning on agent forwarding, etc.) or SCP files back and forth, all the same way that I'd do it with a regular host. But that host doesn't need to be exposed to inbound SSH traffic!
I was also able to get the ProxyCommand working in a Docker container, so I don't have to install the AWS CLI or the SSM plugin on my (Linux) host system, with an .ssh/config entry like this:
Host i-*
ProxyCommand docker run -i --rm -v $PWD:$PWD -w $PWD -u 1000:1000 -v ~/.aws:/tmp/.aws -e HOME=/tmp -e AWS_PROFILE <docker image ref> ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'
* We often use our bastions to do port forwarding, e.g. to connect to an RDS instance inside VPC. As far as I know, we can't accomplish this with just SSH over SSM, so we still keep our bastions.
* User management: SSH over SSM doesn't do any user or key management for you (which makes sense). It establishes the SSH connection and you're on your own from there. We use Keymaker [1] to dynamically create user accounts on the EC2 instances, and populate SSH keys according to the user's IAM profile. This works fine, but maybe there's room for simplification.
* Connecting based on instance ids (`ssh i-...`) can be awkward. It would be amazing if we could alias these somehow, like set up a CNAME that points to `i-...`. I wasn't able to get OpenSSH to follow a CNAME and then use the ProxyCommand. Maybe it's possible, though.