Hacker News Reader

Comment by ttul | Hacker News Reader

I run an email sending service at scale (billions of messages per month, tens of millions of end users, thousands of customers). Most of our software development and operational effort revolves around abuse mitigation. That has been the case for 15 years. It's a cat-and-mouse game with two different mice: the senders, who are constantly trying to figure out how to get you to deliver their garbage; and the receivers, who are constantly trying to figure out how to block it. We're stuck in the middle.

It's hard to appreciate how difficult this battle is when running at scale.

View on HN

Right, I won't disagree with any of that, but I'm not sure how it's related to what I wrote either. Maybe I should have been more specific that I'm talking about hosting your own email, not hosting emails for others, which brings out a lot of other types of problems.

Apologies. When you said "email services" I thought you were implying "email services for use by others". Yeah, you can definitely run your own mail server in 2026 and I think the internet community should always strongly endorse being able to do so. Unfortunately, large email receivers have to make do with imperfect signals when making filtering decisions, and your traffic from a lonely IP that happens to have a bad neighbour might get blocked as collateral damage.

One long term hope: That domain name reputation eventually overtakes IP address reputation entirely.

> billions of messages per month

Just tangential, but maybe that is part of the problem, isn't it? Napkin math tells me that esch person in the world receives every month an e-mail from you, and you're obviously just one of hundres of providers, and only half of the population actially has e-mail... I think you get the point.

E-Mail got to the point where it's actually worse then physical mail to some degree. Physical mail at least has a hurdle for the sender, and it's easier to throw away without even looking at it. The amount of low-quality mail and annoying, unnecessary notifications I receive is just at a level where I really think of dropping e-mail except for absolute becessary services.

What structural changes could we make to improve the situation?

That is such a great question and there is no easy answer. There have been enormous efforts to do better for at least the last 20 years. An entire organization, M3AAWG, was founded for that reason and it meets three times a year, bringing together all the people that matter for making the situation better. It's a great organization and the people are all really smart and awesome. The IETF is no slouch either, coming up with excellent new standards and improving existing ones, such as the recent update to DKIM.

That's about as good of an answer as I can provide: keep sending smart people to the conferences!

It's simple, there's a standard, a new one, which takes into account SPF, DKIM, DMARC, ARC, and even DANE along with upcoming and purposed SPKF, DKIM+, DMARC2, and ARCv4. It should fix just about everything.

Obligatory https://xkcd.com/927/

Hashcash, or BTC.

I always loved the hashcash concept and actually raised our original funding because of it (our Microsoft angels loved the idea of making spamming more expensive, and our Series A concept was tar-pitting to dissuade botnets). In the context of email sending services, we have a modern version of hashcash that we might at some point turn to. If someone can figure out how to tokenize sending at scale, then senders could pay recipients to open their emails by attaching a "tip" to each message.

If even a small fraction of legitimate email recipients altered their mail client settings to route "tipped" messages to their inbox, that would probably suffice to get senders to participate in the scheme. Senders are starved for high quality engagement data. Meanwhile, anything we can do to make spam less likely - on a relative scale - to reach the inbox in comparison to "legitimate" traffic, is a win.

Wow, this brings back memories. Way back in the 2000 I worked on a concept to charge “postage“ to big senders and remit payments to receivers (hosts, not end users). I also looked at hashcash. I ultimately abandoned the project because I realized I would be spending all my time convincing big senders why they should pay and I knew I would hate that job.

Previously I was the spam cop for a big sender and saw up close all the ways our clients would try to weasel around it.

Signed senders?

> I run an email sending service at scale (billions of messages per month, tens of millions of end users, thousands of customers).

Giving you the benefit of the doubt and accepting your claim, doesn't that make you one of the people at least second-order responsible for the current state of affairs in email blocking? It would seem that your company, by dint of your volume, navigates roadblocks that the rest of us (ie. the 99.999% of Internet email servers and their admins), who aren't FAANG et al[1], have to deal with to get our users' legitimate email delivered.

If so, could you perhaps give us a brief explanation as to why an otherwise competent engineer can "follow all the best practices" with their server which has no known compromises[2], on an IP address they have controlled for, oh, let's say a full calendar year, and yet still can't get off those FAANG et al default-deny blocklists, but you can?[3]

A cynic might say that your service had a vested interest in paying for unimpeded access to those FAANG et al companies to get over the bar that the rest of us are unable to vault. A cynic might also say that those biggest of the big email services like it that way, because it drives more users to them at the expense of the rest of us 99.999%.

I'll try to remain open to the possibility that there are aspects of the industry I've not yet had any exposure to, and refrain from chimping out over having my users blocked through no fault of their own.

[1] Yes, I know, Facebook doesn't receive anywhere near as much email as they send, and Hotmail = Microsoft, etc. If I used an accurate acronym I could pat myself on the back for being Technically Correct, while nobody would know what the heck I was talking about.

[2] We shan't digress into a discussion of hardware/firmware/OS/application backdoors nor Snowden disclosures. It's not that hard to auto-install security updates and run a reasonably tight ship with no unnecessary attack surfaces.

[3] Or perhaps there aren't any default-deny blocklists at all, but in fact only much smaller default-allow whitelists? That would be cynical indeed.

Reply on news.ycombinator.com