Not only that, but the listening application must have some kind of exploitable vulnerability.
It would be quite impressive, though, if a vulnerability in an audio driver allowed an uninfected computer to be infected simply by "hearing" the exploit sound!