As a young post doc at the Institute for Advance Study, my Dad knew von Neuman, who according to my Dad, tended to work late at night and work with loud music playing - very much like young hackers today!
"If one has to divide mathematicians, as Poincaré proposed, into two types—those with visual and those with auditory intuition—Johnny perhaps belonged to the latter."
[1] http://www.ams.org/journals/bull/1958-64-03/S0002-9904-1958-...
Not completely similar but somewhat interesting, here's an interview where Feynman admits to not being "great" (for lack of a better word) at music.
https://www.youtube.com/watch?v=eqtuNXWT0mo
“Lot’s of people love music and I never could carry a tune.”