You copy/paste the password it gives you into Pidgin (or whatever you're using it for) and then click "hide" on the box that displays the password, and then you can never see it again. If you trust the machine you're entering in the password so little that you're worried about keyloggers (which copying and pasting might take care of, but I don't know enough about how copy/paste works or how keyloggers work to say), then (1) you should probably not be using that machine to access accounts that you care enough to use two-factor authentication on (because for all you know someone could have installed remote desktop software that would allow them to take control of your accounts the second after you log in, for example), and (2) you can revoke the application-specific password so that they will never work again.
Obviously, using application-specific passwords does make your account less secure, but without them, every single client application, e.g., Pidgin, would need to implement Google's two-factor authentication system in order to be usable. As a user, you are free to choose not to use application-specific passwords at all and get the same security you would if Google had chosen not to allow these application-specific passwords; you just won't be able to use any client applications that don't support them.