Gauss far exceeded von neuman in mathematical impact and pushing math forward. So did many other mathematicians that were contemporaries of von neuman. The same for physics. Von Neuman would not even be in the list of top 10 mathematicians in the 20th Century, let alone of all time. The twentieth century had giants like Kolmogorov, Hilbert, Grothendieck, none of whom were smarter than von neumann, but they made far greater discoveries.

But this just shows that when you are talking about impact as opposed to intelligence, a lot of things other than IQ come into play. I am certain von neumann was much smarter than Gauss, but Gauss had an instinct for discovery that was remarkable. Newton is another example -- someone not nearly as brilliant as Von Neuman (my impression) but had an incredibly deep insight and much bigger impact. They say that Feynman's IQ was ~120, which would certainly be lower than von neuman, but he made a much bigger impact as well.