As we've seen with recent breaches, the last 4 digits of your CC # aren't incredibly hard to find out. "Secret" questions and answers are generally quite poor, in that very few of them don't suffer from laughably small keyspaces or rely on semi-public information. Passwords almost seem like the least bad option.
I get that giving a password to a human isn't a very comfortable feeling, but if you don't trust the CSR to not misuse the password, do you also not trust the developers to not have put in something to grab your password one of the various times you enter it into a web application that they control?