Mathematics may be rooted in that, in a historical or pedagogical sense, but areas of math can certainly be disconnected from physical intuition. Non-measurable sets (e.g. those in Banach-Tarski) and transfinite numbers cone to mind.
I disagree that transfinite numbers are detached from physical intuition since most of the ones you or I could write down can be easily visualized with a few ellipses here or there. But i do think Arnold would consider them marginal players. Perhaps he thought set theory was a formalist distraction from the main of mathematics!