> Then again, I believe even mathematicians are fine with ideas of an empty set, or of a one-element set being distinct from the element itself.

Especially mathematicians. Distinguishing stuff and structure is a common theme throughout mathematics.