Robinson's non-standard analysis is also discussed (and rejected) in the book I linked to. My book argues (among other things) that while all of the constructions of traditional mathematics are logically possible (not very interesting), many of them are metaphysically impossible for any object to instantiate.
For example infinitesimal numbers make no sense, in the sense that there is no such thing (in any possible world) as a positive number that is smaller than every positive real number. Of course, you can definitionally augment `(R+, <)` with formal objects `u` s.t. `u < x` for any `x ∈ R+` (where `R+` designates the positive reals), and then close these under the field operations `(R, +, *)` (I assume this is how infinitesimals are constructed). But that doesn't mean you have written down a model of actual numbers (which include the so-called "infinitesimal" numbers), since there's no such things as infinitesimal numbers.
You could argue "well, maybe infinitesimals aren't actually numbers (and perhaps more strongly: the theory of infinitesimals fails to have any metaphysically possible model), but they are still useful to us". For example: calculus is surely useful to us, and calculus can be built over the "infinitesimals". I don't have anything against this. But many people studying non-standard analysis incorrectly think that because infinitesimal numbers are being formally studied, that must mean there are good arguments out there for their actual existence.
Another incorrect dogma from traditional mathematics has to do with infinities being treated as numbers. Infinities are not numbers, for there is no such thing as a number which is greater than every natural number. There are countable (and even uncountable) collections (that is perfectly coherent). But that doesn't mean infinity is literally a number. Yet in traditional mathematics, it is merely asserted that infinities are numbers (and that indeed there are an infinity of these infinities, that can be ordered by size).
Even the ordering relation is asserted without argument. For example, let `N` be the naturals and `P(N)` be its power set. In traditional mathematics, we say that `|N| < |P(N)|` since every any injective function from `N` to `P(N)` fails to be surjective. Sure, you can define `<` to satisfy this. But that doesn't mean, in reality, the "less than" relation does satisfy this! It is merely asserted, without argument, that `|N| < |P(N)|`, when it would seem to most smart, reflective people (before they are corrupted by years of traditional mathematics training) that `|N|` and `|P(N)|` aren't orderable in the first place, since they are both infinite collections (the former being countable and the latter being uncountable), and infinite collections can't be ordered by their size (in the "less than", "greater than" sense of ordering we use with actual numbers). Of course you can define an ordering on infinite collections using the surjective relation, but that doesn't mean it's mapping onto the actual "numbers ordered by size" relation that we had in mind before we started studying infinite collections. Perhaps this is why beginning students studying infinities in analysis find it so mind-boggling: they're being taught a bunch of forced definitions/constructions which don't actually map onto any real (or even metaphysically possible) models.
When you take the traditional mathematics route literally, you run into all sorts of absurdities. For example, if you treat ∞ as a literal number, you have to patch up the field operations on it. You have to start saying: "∞ + 1 = ∞", which implies it's not the case that "∞ < ∞ + 1". You can find several other examples in that book I linked to.
The original Cantor's function (a function that is continuous everywhere, but has zero derivative almost everywhere) that OP linked to strikes me as something that fails to have a real model. I don't doubt that such a function is logically possible, as constructed. But it likely means our definitions of "continuous", "derivative", "function" (or other such mathematical constructions) are failing to map onto the abstract objects we set out to when we started studying them in the first place.
Sort of like how, in probability theory, you can have events which are possible but nevertheless have probability zero (classic example: throwing a dart at the real number line means that the probability it will land on any particular real number is zero). It's totally logically coherent to set things up this way, but that it is logically possible for this to happen means our foundations/definitions/models are all screwed up (though, I admit, still pretty useful nevertheless).