I think there's a reasonable argument to be made that mathematical constructs are
part of the universe.
David Deutsch has a method for ascertaining whether something can can be said to exist or not - ask whether it "kicks back" when you interact with it, in the sense that simulating the response of the thing you're considering in a totally convincing way would involve an effort as large as building a new universe for that thing to exist in.
Rocks "exist" because, if you wanted to build a totally convincing simulation of a rock, you'd need to include all of modern physics as we currently understand it.
Other minds "exist" because simulating them convincingly would basically require you to construct a true articificial intelligence (strong AI).
Mathematics "exists" because giving someone the genuine experience of doing mathematics when they really weren't would involve a simulation of almost unfathomable complexity.
There is a very real truth in the statement that '1 + 1 = 2', or the statement that there are arbitrarily long arithmetic progressions of prime numbers, or any number of other mathematical results. The world of mathematics is a very real part of the universe, that "kicks back" by constantly surprising us when we think about it.
So I don't think the article, which is very well written and researched, deserves the middlebrow "Um, no" scorn that you treated it to.