Yes: https://www.ibm.com/docs/en/cryptocards?topic=4768-overview
Yes: https://www.ibm.com/docs/en/cryptocards?topic=4768-overview
If I wanted to store an important long term key in a secure facility, I would worry, first and foremost, about software attacks, attacks doable over a network, malicious firmware attacks, and maybe passively observed side channel attacks. Physical attacks would be a rather distant second.
Now if someone evil-maid attacks the HSM itself, that’s a different story. Any good HSM should resist this, especially one found in a portable device. And this is because you can steal an entire important corporate laptop or other portable device without necessarily raising an quick alarm, whereas I have trouble imagining someone walking off with the HSM out of an IBM mainframe or with an AWS HSM without the loss being noticed immediately.
(To be fair, in the mainframe case, some crusty corporations seem to have a remarkable ability to fail to notice obvious crypto problems like their public facing certificates expiring. But a loss of an entire HSM from a secure large cloud datacenter will, at the very least, immediately trigger “elevated failure rates” or whatever they like to call it…)
It depends who is the attacker. There are countries (western democracies) where the police regularly "visits" datacenters.
The adversary will show up and badge in just like everyone else. They might have worked there for 20 years, or they might be an outside repair person or external consultant.
They will definitely fit in. They're supposed to be there.
It will be the most normal thing in the world. And you may never know their real purpose.