Edited to add:
It does depend on what you are doing with the numbers. Focussing on the storage side is missing the point. It’s about the ambiguity. In general I’ve found that anything other than integers is a mistake.
For example, *555 is not a valid phone number. You can’t put it in a tel: URL and you can’t dial it or send a text to it. It will work only in limited situations. If you want to store an extension it should be a separate field.
Once you realise that phone numbers are functional, almost exactly like IP addresses, you realise that storing them as integers has benefits because that way you literally are unable to store useless data.
Once you get in the habit of using only e164 numbers, the phone number becomes usable in almost any context, and you can make functional assumptions about it.
As an aside, it’s much easier to create fast and small prefix indexes on integers, but that’s a different story…