As an example, an offline PKI root backed by a picohsm would be fast enough, given intermediates aren't typically minted that frequently.
If you're sure you never do a plaintext backup or anything else that might leak the key, then it may not matter. But it is a decent defense in depth measure for high assurance operations.
I heard a story once about a CA (public? private? not sure!) that had an airgapped root CA in a basement lab, only for an enterprising admin to run an Ethernet cable between their desk and the basement lab to save themselves a few trips... An online HSM may allow effective leakage (by allowing arbitrary signing operations), but hopefully would prohibit export by a crypto user and retain audit logs of all operations to identify scope of the compromise.
At some point, you end up reinventing an HSM from first principals. :-) My 2c.