What's 2-nd factor password? Well, basically yubikey stores just long text string, and another, shorter string, is stored in my brain. When i login i enter short string, then press yubikey.
To steal my data you don't only need to steal yubikey but also get my part of the password from me.