Hacker News Reader

Comment by pmlnr | Hacker News Reader

I genuinely fear the future of email. Email is still the only piece of communication you can own, from top to bottom, from running the service to owning the domain[^1] it runs on. You were able to send anyone email, regardless of rank, location, social status.

Google (and recently, Outlook) is taking all of it away. It's putting mail from people not on your contact list in spam[^3]; it's by default blaclisting IPs within certain range[^2]; now it's bringing it's own format as well.

Is this embrace, extend, extinguish, Google style?

[^1]: let's not get into the problem of DNS never really being owned, only rented, for now

[^2]: https://wiki.hetzner.de/index.php/Microsoft_Blacklist/en

[^3]: only experienced it, friends did as well, but I don't have proof.

View on HN

I run my own mail server and Google - for once - seems to play reasonably nice, it is Microsoft and Apple that are the harder cases, in part because they outsourced some of their vetting to third parties.

I've been mulling switching to Google or some other email provider for a long time and I still feel that this is the last bit of corporate independence that I'm willing to give up. Our mail is ours. I don't care of the counterparty is using gmail or whatever other flavor of cloud services they care for but running your own mail server should be a first test of whether or not you are an IT business or not.

It borders on the ridiculous to have to outsource something as essential and confidential to a commercial provider in another country who only entered the scene about a decade after us, especially because I see those commercial providers as the biggest part of the problem to begin with and this feels like rewarding them for their abusive behavior.

A snail mail analogy escapes me but it would involve sending your post all the way around the planet first before dropping it in the mailbox in the next town over, with some random elements for delivery or non-delivery thrown in for added amusement.

All this gives rise to some frustration on occasion, it is no fun to have your legitimate email classed as spam for reasons not under your control but so be it, I'll take that over giving up autonomy.

Google 'AMP' for email is yet another step in the wrong direction and I'm sure that it will get a lot worse before it will get better but at some point I'd hope that people will wake up to the fact that all this consolidation of power is a negative thing.

The core problem jacquesm is that we have n-to-n communication with email, no fiscal burden to fire up an email address, and the asynchronous nature of the platform means that its just the perfect target for spam and social engineering.

If you want to fix email you can, but it would be better to fix the internet first. Communication should be over an encrypted web of trust. Once you have that then email easily bolts right on. The only thing holding back this change is that the business model of the internet is rooted in surveillance because the web is largely monetized via advertising.

In the mean time semi-bridged walled gardens are going to be the answer to the growing hostilities on the internet.

I seriously don't understand why you are using such a fear-laden apocalyptic language to describe minor technical issues. Email really works for me pretty fine. And internet too, btw.

> I run my own mail server and Google - for once - seems to play reasonably nice

Wait until Google does one of their fun blocks on you for a couple of weeks. They don't have humans involved and just don't have to care.

Google/Microsoft didn't kill email, spam did. Google/Microsoft then salvaged from the abyss something largely similar to email, but with some limbs all but severed.

Spam put centralised commercial e-mail ( ISPs and web-mail ) into critical condition until the web-mail hosts reacted to save themselves.

But as for the little folk who are less of a target: I have run my own mail server for 18 years and can't imagine I've received more than 100 spams in total. And that is without any defences other than SPF.

ISP e-mail, well that is certainly spammed to death but probably for the best as it was only ever a hook to keep people trapped in sub-par services.

I also run my own email server, and I've received 400+ spam emails this year alone, just from the eBay leak of 2014 :) Thankfully I use a different address for each service, so I blackholed that to the spam folder and created a new one.

I have a personal domain on Gmail, and my spam folder currently has 105 messages in it from the past month. Two are mis-filed mass mailings and one is from a mailing list that I think I was once legitimately on, but which has refused to unsubscribe me. The rest are from russian women who thinks I'm sexy and a few from the Nigerian central bank (that's apparently still a thing, who'd have thought).

These are the emails that are not related to me being on a centralised commercial host, and they all made it past Googles ample defences.

That tells nor about the state of alleged defenders than state of email. 2009 vintage spam filter (CRM114, OSBF, extra weight on headers) is easily much more accurate in all senses than GMail alleged spam filter. The problem with GMail is it uses some generic idea of unwanted mail that never correctly applied to the mail you receive in the first place - to save space on personalised filters and speed up processing. Oh, and to extract global statistics.

* http://crm114.sourceforge.net

I usually receive between 5 and 40 spam mails per day (best and worst day of january 2017), manually sorted.

That's with SPF, DKIM, DMARC checks automatically rejecting emails. SpamAssassin and custom ML filters to presort.

Email is already fucked. In my experience Gmail often failed to receive emails from services like Yandex and Zoho. Plenty of people here that setup their own servers, and know what they are doing, getting randomly blacklisted.

Maybe it's fine if email is not central to your life or business but if there's even a, let's say, 10% chance of my emails not getting delivered to Mr Google I have no other option than to bend the knee.

On the other hand, I’ve had only good experiences with self-hosting my email.

I’ve never had issues with being blacklisted or anything.

But I’ve been sending with SPF + DKIM + DMARC + ADSP all configured and enabled since day one, and with TLS transport for all sent emails.

That usually gives a major boost in trustworthiness.

This matches my experience as well. I rarely if ever get sent to a spam folder. And afaik, I've never been blacklisted after ~5 years.

I have everything properly set up, and wasn’t blacklisted once in any service since 2002.... until 3 months ago, when Google decided to blacklist my domain, but only randomly (e.g. mails sent to same person on same day get delivered to inbox or spam seemingly based on a coin toss).

The only clue I have so far is that in some discussion, it was mentioned that google penalizes a domain of a lot of different emails from it get forwarded to google - and I do have my catchall forwarded to a gmail box.

We now live in a world where google can make you and your communication channels disappear, and they don’t really answer to anyone.

Have you tried to check https://postmaster.google.com/? You _may_ get an insight on what went wrong those 3 months ago

That requires hundreds or thousands of emails from your domain per day to show anything - which is useful if you run a major service, but for the use case of "every person hosts their own email", it’s not exactly useful.

Well, you're right, though I've seen data in there for domains with about 10k emails per month.

Still, that's probably way over the home email setup threshold.

Unless your IP address is deemed being in a bad block. Then all of the above is not even checked (on outlook.com, that is).

It's a tragedy. The people who designed SMTP went to insane lengths to make sure mail was delivered. The people who initially implemented it did the same. Then the "free" mailbox providers fucked it all up. I host my own email, and I now have no clue as to whether it will be delivered.

Google’s style is “develop, control, prioritize, monoplize”.

It's like Microsoft in the 90s/00s with embracing and extending.

> Google’s style is “develop, control, prioritize, monoplize”

“develop, control, prioritize, monoplize and sunset”. Don't forget the last step.

Rinse and repeat.

Last is the only consistent though

The only reason they can do this is because of the abuse of email. Getting email from everyone means getting email from every spammer every day. Anyone could stand up their own email server, or grab a new address from their ISP or Yahoo or a zillion other free/cheap email hosts. But Google dominates because they reduce the amount of crap their users have to put up with. Blacklists are a feature that users seek out.

We've crowdsourced our adblocking lists without much of a trouble, so I don't think that we would have some problems with doing the same for spam accounts (or domains).

On the other hand, you can't easily import an email blacklist as you can an adblock list, so there's a problem that will never get solved in popular email hosting providers.

Whitelisting certain IP ranges/domains/whatever will never get us to our goal. Making sure the spam filters are 100% accurate is too much of a task for one company (heck, just this week, I missed an email from the embassy because the Zoho filter thought it was spam).

We've proven that crowdsourcing works already, but there are some nitpicks to be solved (what if an email account gets hijacked and later on returned to their original owners?), the biggest one of which is for Gmail, Outlook and others to support the import of such lists instead of mining data (something they're unwilling to do).

And yet I still use email because I am no longer inundated with spam thanks to the likes of Google and Microsoft. Without them I'd have given up on email altogether.

I run a mailserver with bogofilter, which, with a few months of training, is surprisingly effective. Before that, dspam was also very effective. Spamhaus makes a decent effort on providing blacklists. I don't see what google and M$ is bringing to the table, apart from pain for indie (still validly configured with DMARC, DKIM, SPF) mailservers.

> I don't see what google and M$ is bringing to the table

It depends on which perspective you're using. A company paying staff to host private mail infrastructure? Individuals who aren't in a position to take on that workload and/or those who choose just not to?

Google and Microsoft are providing fairly effective spam and malware filtering to the masses with options to classify emails using at most a couple clicks. That's pretty much out of the box experience and doesn't involve setting up and training Bayesian filters for months.

For the longest time Gmail offered email threads that were more intuitive than implementations found in other email clients with sent messages presented in related correspondence. Although other clients and webmail services seem to do that now as well.

Using free services is a trade-off but individuals who choose to partake clearly benefit in some ways.

For me they bring to the table everything you just mentioned, except I don't have to do it.

> run a mailserver with bogofilter, which, with a few months of training, is surprisingly effective.

I don't use gmail but I have an email address from Microsoft (which is not m'y main one) : outlook.live.com does a good job filtering newsletters or promotionnal emails, or other terrible marketing practices, from individual ones, but it's not espacially performant when it comes to dealing with spam or scams. On this address I have a «please log in to your PayPal account for safety reasons blah blah blah» or «come buy our absolutely legal drugs to boost your performances» every three or four legit emails.

Google and Microsoft certainly did not help, but let's not forget that many of these features were made to combat spam which got more sophisticated over time. Having all mail delivered all the time, trusting everybody on the web and so on was fine when internet was for enthusiasts. In today's environment, however, plain old e-mail has insufficient protection and overly optimistic design.

Bayesian filtering is surprisingly decent, if it has a feedback loop. Trust by default, and let users train spam filtering, based on a generic, prepopulated filter database.

Did not help? I am old and remember email before Google solved spam with Gmail. So they most certainly helped.

Google didn't "solve" spam at all, they are just playing rough. The jail sentences for spam network operators, the the shift to ddos/mining/etc of botnets, and user awareness, bitten by spam helped, but not solved. You probably used a crappy provider, with no feedback loop for spam before.

Did not help keeping the e-mail decentralized. They did help against the spam. Well, at least google did. Microsoft hotmail was doing spam filtering almost exactly the opposite way it should work for so long that I have completely given up on it.

I experienced [^3] as well, but every time I dug into it, it was due to misconfigured mail servers like missing SPF records.

Unless somebody has proof for it, I guess attributing it to a poorly explained AI black box instead of malicious intent is more sensible, similar to the "Facebook is listening to conversations and showing ads accordingly".

Why is it the only form? You can certainly run your own IRC or Jabber server or something. I don't understand why you think only email can be run top to bottom.

Unfortunately email became stupid a long time ago. Partly thanks to braindead clients like Outhouse and partly thanks to Gmail. Remember when Gmail came out and they advertised "threads" as a feature? Yeah... we had threads before Outhouse et al. proliferated enough to make them useless.

And don't forget the way those clients make you "quote" previous messages.

Free software mailing lists still make email nice to use. Simple, plain text and intelligent.

Threads that don't play nice with any other e-mail client, where Thudnerbird drafts would just fill up your thread history to the point it wouldn't make sense.

And terribly broken IMAP support that wouldn't use IMAP labels, but instead transfer a separate copy of an e-mail to multiple folders if it had multiple labels.

I thought Gmail was amazing when I first got it, and by 2012 I started running my own e-mail server when I realize it was a usability nightmare.

It's pretty confusing to refer to Microsoft Outlook as "Outhouse."

Very Stallman-esque.

https://stallman.org/apple.html

You can't have you IRC server and talk to another IRC server with it. I could still pull a BBS up but apart from FiDo, they don't talk to each other.

Yes, I know about Matrix - doesn't have critical mass yet. I know about XMPP, but most of the XMPP servers don't allow other domains to talk to them.

I agree with you on the dumbing down though.

> I genuinely fear the future of email

Email is already the worst form of communication. Spam has basically made it useless.

You are literally a decade late. Email has been dead for a long time. You're right that Google EEEed it.

As long as the majority of people are relying on emails (regardless of the use case), it ain't dead.

Even if you don't use it to communicate to other people, you're still relying on it (for example, you forgot some random password), making it impossible to get rid of.

Nothing replaced it, and nothing will replace it for the next decade. Not all people on the Internet have a Facebook account where they let anyone in their friends list, and not all people on the Internet have a phone number that they are willing to give publically (for WhatsApp/Signal/any-mobile-first-IM-solution).

If you try reaching me online, you won't find my Facebook account on a search engine (nor on Facebook, unless we have some friends in common), and you won't find my phone number easily (unless you already have me on Facebook as a friend). You'll find my email address easily though, and that's why it'll remain relevant.

Same could be said for pretty much every GitHub user (that's why email addresses are right on your profile by default), every academic that has his own website, most of HN users (just take a peak at random profiles) etc.

I hope email disappears. It has no security, bad tooling and software, weird protocols. Ip reputation has long been a thing in email.

You can own IRC as well. I feel like your 3 cons are my three pros.

> It has no security

Little communication software out there can be made as secure and resilient as email.

Two people out there can run their mail servers on a laptop or a raspberry pi. Email is easily encrypted with TLS and GPG on top.

The initial setup for this could take less than 2 hours.

Bonus points: everything could be run from a *sh shell, 1 CPU core and 128 RAM is quite enough.

> bad tooling and software

Postfix (and HAProxy for that matter) are some of the best pieces of software I have the pleasure to work with. Very good docs, I personally haven't seen them crash and have a way to never take you by surprise.

Resilient and secure compared to regular snail mail, you must mean.

You're dangerously wrong about the security of email...

If email is so insecure, how come almost every service uses email as a recovery tool?

There must be hundreds of thousands of very valuable accounts that could be accessed if you had access to someone's emails, so why doesn't that seem to happen?

You two are totally missing to use the threat model approach to security.

Email is "secure enough" for common people, whose threat model isn't high. We have Google and Microsoft to thank for that primarily, since they're the ones that pushed 2-factor auth, encryption in transit (HTTPS) and other features (that later on got implemented by all the email providers). Those features themselves would mean nothing if they weren't incorporated in the biggest free email hosting solutions.

Email is "completely insecure" to those who can't trust a third party (like Gmail). It has GPG on top of it, which is nasty to use from a user's perspective. Meanwhile, even if you do all the things perfectly and never screw up, you're still not getting the same level of protection you would get from using Signal (as a solution that doesn't retain any metadata), whose user experience is out of this world compared to GPG.

Is email secure enough, or does Google/Microsoft running people's mail servers make it secure enough?

I think it's the latter.

I would say it's in between.

Email by itself (as a protocol) is far from perfect, but you can have other mechanisms to improve on top of it where it falls short (while some other things, like metadata retention, are deal breakers). You can still host your own email with almost all the bells and whistles offered by Google/Microsoft, so you have that going for you.

On the other hand, even if you have the greatest and most secure emailing server imaginable, you would still be communicating with others who don't use it, and you're relying on them having some strong security mechanisms as well. Therefore, it's important for Google/Microsoft to make these improvements on top of emails as well.

So, if we're talking about email as a protocol, not secure enough for 21st century.

If we're talking about email as an end product, then yes, it's secure enough (under the assumption that you're using some well known email hosting service).

Reply on news.ycombinator.com