> in general people look funny at you when they see you with an yahoo email address instead of a Gmail
That's the only problem I have after migrating to Fastmail: it takes serious effort to make people over the phone understand that I have an email in a different domain. I can spell it letter by letter, but people often still can't figure just where this funny thing I'm dictating should be put. There's a problem with @ sign, too - somehow, people tend to forget it exists. Well, it exists to separate user from domain, and since they don't know the domain part exists, they don't need to know about the separator I guess.
TLDR: if not on GMail, you need to make a habit of sending the address in an SMS, forgo all hopes of giving it to someone over the phone.