I remember reading a similar anecdote in the book Turing's Cathedral, where it suggested that von Neumann had a particular preference for loud marches. It also makes me think about what Stanislaw Ulam wrote of von Neumann[1]:
"If one has to divide mathematicians, as Poincaré proposed, into two types—those with visual and those with auditory intuition—Johnny perhaps belonged to the latter."
"If one has to divide mathematicians, as Poincaré proposed, into two types—those with visual and those with auditory intuition—Johnny perhaps belonged to the latter."
[1] http://www.ams.org/journals/bull/1958-64-03/S0002-9904-1958-...