I agree with your assessment on facial recognition and fingerprints, though the biggest problems with the fingerprint authentication are the lack of ubiquitous sensors and privacy issues.
But password manager software is better than nothing ;)
Is there something that makes a smartcard based USB key work differently when held by a person other than its rightful owner?
My impression of them, perhaps incorrect, is that they are akin to Hanko ( http://en.wikipedia.org/wiki/Hanko_(stamp)#Japanese_usage ). Basically a system of using possession combined with a fairly unique set of data as a means of distinguishing and authenticating its owner. But in practice its far less than perfect.
Nothing is perfect of course. The risk/threat is very similar to someone stealing your phone and has somehow gotten to know your master password to your password management software. However, password management does not really solve neither the usability or the security problems of usernames and passwords ... although a good piece of software can certainly help.