As far as I know, SSH over SSM doesn't do anything regarding user management. It just establishes an SSH connection. Management of users on the host, authorized SSH keys, etc. is totally out of scope for SSM.

So if you already have access control setup on your host, then SSM doesn't do anything to undermine it. If you don't have it, you'll still need to add it.