The capacity to grant access as a specific remote user is present without certs as well right? The typical authorized_keys file lives under a user directory and grants access only to that user.
You can also make all the certs short-lived (and only store them in ram).
What I've done is generate a cert for the host(s) the user needs, for the time-span they need (subject to authorization logic).
If your endpoints can securely and reliably reach a central server, this gives you maximum control (your authorized_keys HTTPS server can have any custom business logic you want) without having to deal with certs/CAs.