Additionally, this will make it look like you're a spammer.
You could block SMTP on almost all end user networks with no one noticing. 99.9+% of email client submission is HTTPS.
Source:
I doubt it's that much but with the same logic you could also ban HN, SSH and basically any protocol thats not https "with no one noticing" because 99.9+% doesnt use it.