Encrypted filesystems are necessary for #2 to be safe, but merely move the problem of #1 up one step in the chain to OS boot rather than app boot. #3 isn't necessarily safe as the HSM could be used to decrypt data when an attacker has gained access and is undiscovered, unless you use it to issue a temporary decryption key at OS/app boot, at which point you're effectively back to #1 again.
In practice, if you're a server application author, either use the secure credentials storage service offered by your cloud provider or store your secure tokens encrypted on-disk with the decryption key either stored in a local config file elsewhere or entered at app startup.
There have been several issues reported around this, but no one seems to have a good proposal for a solution.
I've also discussed this with security auditors from Cobalt (because they flagged it as an issue), but they also did not propose any solutions (other than using httpOnly cookies instead of tokens, which doesn't really address the issue).