The promise of HSM, TPM and smart cards are that you have a tiny computer (microcontroller) where the code is easier to audit. Ideally a sealed key never leaves your MCU. The cryptographic primitives, secret keys and operations are performed in this mini-computer.

Further promises are RTC that can prevent bruteforce (forced wait after wrong password entry) or locking itself after too many wrong attempts.

A good MCU receives the challenge and only replies with the signature, if the password was correct. You can argue that a phone with a Titan security chip is a type of TPM too. In the end it doesn't matter. I chose the solution that works best for me, where I can either only have all keys in my smart card or an offline paper wallet too in a fireproof safe. The choice is the user's.