In school it was explained through combinatorics, where the factorial among others tells, in how many ways you can order your set. And an empty set has just one, but it has one, order in which it can be arranged.
Now that I am writing this, I'm losing confidence, that this explanation is mathematically correct..