If you want a cheap HSM, there is also nitrokey:

https://shop.nitrokey.com/shop/product/nitrokey-hsm-7

I've bought one, but didn't use it yet. I had ideas to use it to test an HSM + pkcs11 OpenSSL engine setup in a personal project (RFC3161 server), but I didn't found the time to play with it yet.

However, the standard tooling to interact with a P11 device under Linux is a bit of a mess, you have tooling from gnutls, p11-kit and OpenSSL (kind of actually, they kind of broke arguments passing, for example the URI of the key not being a valid path, when using external engines).

Also, if you want to play with an HSM without a physical HSM, softhsm is for you (https://www.opendnssec.org/softhsm/).

That price point needs to come way down. Yubico thinks they're competing with established HSM vendors, but they're really competing with their own products. If I have a small business and am considering a hackish YubiKey-as-HSM for $40, it may be worth it to spend around $100 on an officially supported YubiHSM instead.