I'm pretty sure asking for references from provided references and contacting them without getting permission from the applicant is illegal. Also most people would feel there was a serious invasion of privacy I think.
Certainly it can violate any kind of implied discreteness in the search. I would think that you'd need permission, but then it might not work as well. The best you could say is, "I think we both know X, can I ask her about you?"