The private SSH key never leaves the smartcard, not even during authentication. It is not exposed to the OS or any process, at all. You can't extract it at all (maybe the NSA can, who knows). The actual authentication takes place on the token, not in a process on your Unix system.

The only thing that the malware can do is issue an authentication request while the token is plugged in. That's all. If the PIN is not cached, you'll be prompted to enter it, and you'll be like "why is it asking me to enter the PIN?"

Maybe they could run a spy debugger on gpg-agent, but again, this would not give them your private key.