The key stays on the card. The card will do signatures and key generation, and also holds login/etc credentials. It works basically exactly like Malka says: bytes go in, signature comes out. It can likely do encryption as well, but you'd probably use it to generate a temporary key and then to sign the encrypted results, since the processor on cards is relatively weak.
0: http://www.acs.com.hk/en/products/17/acos5-64-cryptographic-...
And when I'm spending a month or two in Asia, for example, I have no illusions that I could possibly get another mailed to me, if it broke or was lost.
These are great in terms of security, and should be an option for people who need it, but shouldn't be obligatory (I wouldn't use this for my bank if I had a better option).