The Mathematician (1947)
www-history.mcs.st-and.ac.uk
www-history.mcs.st-and.ac.uk
My tl;dr: Von Neumann sees mathematics as being primarily inspired by empirical problems. The further mathematical theories veer from their empirical origins, the more "baroque" or "degenerate" they become.
It would be interesting to know what inspired him to write this. Maybe something equivalent to, e.g. modern string theory, which was pissing him off.
Google found this in "Prisoner's Dilemma/John Von Neumann, Game Theory and the Puzzle of the Bomb" by William Poundstone (page 29):
"As the same time he insists that the best mathematics is usually inspired by practical problems. This can be read as a defense of game theory (among other things) to those fellow mathematicians who deprecated it as an applied field."
So perhaps more out of an irritation with condescending/dismissive pure mathematicians than an irritation with overly abstract theories per se.
Hopefully someone who knows more about Von Neumann will stumble along these posts and provide more insight.
I really didn't get the impression that this was an attack on "very pure" mathematics. Perhaps an attack on the notion of absolute/universal rigour. The stuff about degeneracy appears almost as an interesting aside.