Can you point to anyone like him in the recorded history of the last thousand years? There may well be some whose lives were unrecorded or of whom the records were lost.
But this just shows that when you are talking about impact as opposed to intelligence, a lot of things other than IQ come into play. I am certain von neumann was much smarter than Gauss, but Gauss had an instinct for discovery that was remarkable. Newton is another example -- someone not nearly as brilliant as Von Neuman (my impression) but had an incredibly deep insight and much bigger impact. They say that Feynman's IQ was ~120, which would certainly be lower than von neuman, but he made a much bigger impact as well.
Another mathematician that reminds me of Von Neumann is Euler. He also memorized long passages and could do complicated calculations in his head quickly.
A quote on Euler from wikipedia:
“He was able to, for example, repeat the Aeneid of Virgil from beginning to end without hesitation, and for every page in the edition he could indicate which line was the first and which was the last even decades after having read it”
He famously recounted his inability to derive Heron's formula for the area of a triangle when he was a teenager (despite realizing that such a formula ought to exist via conceptual reasoning), and seems to have subsequently kept an unbalanced set of talents in the same vein.