I completely agree with this. The trouble also is that "real" recursive problems tend to be _very_ hard: an example from physics would be magnetohydrodynamics where a plasma moves in accordance to an applied external B field, causing a current to flow, which generates a B field that changes how the plasma flows...you get the idea.
Everyone knows that you don't _really_ want to try doing 123! using your "my first recursion" code in C. Similarly everyone knows that factorial gets pretty big pretty quickly – I remember finding out the limits of calculators in school by finding at which point x! went from "big number" to "error". It's pedagogically useful for those things and you can see (a) if your answer is right, compared to a BigNum library, and (b) how long it takes. There's a whole can of worms you can go down, from Sterling's approximation to a discussion about time complexity, ints and IEEE 754, and stack overflows. You could then go waffling about, e.g. Haskell and lazy evaluation and functional programming and so on.
Factorial isn't a good problem but it's common enough that students will have heard of it, deep enough to be interesting, and used so frequently as a programming example that if you see a recursive definition of factorial in another programming language you get that they're trying to teach you about recursion. It's a good teaching tool for all those reasons.