Usability issue aside. ... Using SMTP as the only login factor sounds very insecure to me.