Hacker News Reader

Comment by PinguTS | Hacker News Reader

I completly agree with you. While Hetzner is my actual neighbor as their headquarters is right in the neighboring town, I still use a server at very small scale provider. I have no problem with my email server. I receive some spam here and there, mostly from Russia. But I immediatly block the according IP addresses for some time.

For years I avoided to use any external service to decide whether its Spam or not. But about 2 years ago I started to rely on some of the external Blocklists.

Till today I have no problem sending Email. Even as I don't use DKIM or DMARC.

View on HN

If you are using external blocklists, you are perpetuating the problem here. Small mail operators end up on blocklists through no fault of their own, and it sounds like your server would reject them just like the big servers do.

You don’t have to use blocklists to block messages.

I use blocklists in my self-hosted setup but only for the purpose of adding header fields that my Bayesian anti-spam filter can use to classify messages. I don’t reject anything out-right, aside from attempted spoofs of the domains my server is authoritative for. Everything is received— it just may end up in an “Unsure” folder if it seems too shady for the filter to put in my inbox.

That sounds like a better solution and not at all what I would assume someone means when they say that they use blocklists to filter out spam.

Not using block lists will inundate mail servers with spam. It's a necessary evil.

And for most people, using a large mail provider is the necessary evil. I don't see much difference between a small mail provider using block lists and a large mail provider using the same block lists + their own proprietary lists and metrics.

Both result in other small mail providers getting blocked for arbitrary reasons with limited recourse.

Using large mail provider is not necessary evil in my book. It is just so much more convenient than self hosting. Today, with ready-made containers and stuff we start having a fighting change, but the moat is possibly too big to fight with email falling out of favor

I don't use any kind of block list on my mail server, as I find the concept to be fundamentally flawed. I accept all incoming email as long as basic sanity on the connection is met and then apply bayesian filtering after the email has been accepted.

I get just about no spam at all (<10 per month, maybe).

One spam every three days is insanely high IMO (compared to what you will get on gmail)

I get > 1 spam per day on gmail.com.

But I don't find that burdensome at all, in fact I'd prefer that gmail let more stuff through to my spam folder instead of swallowing things that it misclassifies, because I have seen several instances of lost legitimate email, something I still don't understand how is deemed acceptable to the people who wrote it.

But I need to vote with my feet instead of complaining.

> One spam every three days is insanely high IMO (compared to what you will get on gmail)

That's funny since on gmail I get tons more spam than on any of my self-hosted domain addresses. What's worse, I see emails in gmail misfiled as spam when they are not, which is far worse. I don't ever see that happen in my self-hosted system.

I use block lists, fail2ban, gray listing, most of rspamd's widgets, and Bayesian filtering. Still getting 10s of spams a day just to my accounts.

Any guess on why people's experiences with spam are so diverse? I self-host and get fewer than 10 true spam emails per year (not counting marketing newsletters).

> Any guess on why people's experiences with spam are so diverse?

That would be an interesting research project. The domains getting most spams on my servers are the ones that are old (20+ years) which I guess makes sense.

Checking the rspamd logs for the last month gives just shy of 8500 emails with what I'd consider "definitely a spam" score. There's probably another 1000-1500 sneaking under that.

You mentioned receiving 10s of spam per day - are they being classified correctly and put in your spam folder or are you seeing them in your inboxes?

My mx hosted with hetzner also runs rspamd. Of the 32k mails received in the last month, 40% were rejected (postfix DISCARD, so the sender sees the mail as accepted but its sent to /dev/null - this only happens to mails scored very highly as spam, or sent to a spam trap address), 10% were greylisted and 1% were delivered to the spam folder.

So I'm also receiving 10s of spam per day, but they're all delivered to my spam folder with rare errors.

With the exception of an issue delivering to AT&T recently, I haven't had any outbound deliverability problems in a few years, but then again I don't send very much mail at all - perhaps I'd have more trouble if I did but most of my mail is incoming.

> are they being classified correctly and put in your spam folder

Mostly. There's probably 3-5 spams a day which get into my inbox; also 10-15 a day that wrongly end up in the "maybespam" folder (generally bulk that I'm not worried about seeing 100%.)

> That would be an interesting research project. The domains getting most spams on my servers are the ones that are old (20+ years) which I guess makes sense.

The old lists may be more plausible.

Some spammer once associated 'Robin Bennett' with my email addresses at some point 20 years ago and kept reselling it. Never used that name, didn't even know Bennett was a surname. I assume they linked me up with that name to fill up empty cells in their list and make it look more plausible.

It's a good way to filter spam. The last decade almost all spam calling me 'Bennett' is from some US political group, which mostly reminds me that it will be a long time before anything GDPR-like will pass in the US..

But almost all spam goes to that old 20 year old email address I don't use. My current email addresses are much cleaner even after 10+ year of use

> > Any guess on why people's experiences with spam are so diverse?

> That would be an interesting research project.

Indeed. When you say get spam, do you mean pre or post-filtering?

Over the last 30 days (I don't keep them longer than that) I've received 43 spam emails which were sent to the spam folder (so I wouldn't ever see these, other than because I went and looked now for the sake of this discussion). In the same time period there was only 1 spam email which was missed by spamprobe and made it into my inbox spool.

There's a fair amount of would-be spam that gets blocked during the SMTP transaction due to things like bogus host in HELO, etc. I don't keep any stats on those, I did at one time but it was too much noise.

My email isn't secret, it should be on every spammer list I'd guess. It has been the same since the mid 90s and is all over usenet, email list archives, websites, etc and I've never made any effort to mask it.

> Indeed. When you say get spam, do you mean pre or post-filtering?

All my numbers are post-rspamd. There's still sieve after that which does the "is it scored more than X?" to redirect into spam/not-spam.

> There's a fair amount of would-be spam that gets blocked during the SMTP transaction

Yeah, the blocklists and protocol strictness rules definitely cut down a lot before it even gets to rspamd.

I thought self hosting is for aliases? Shouldn't those fix spam for good unless you publish your address globally?

> I thought self hosting is for aliases? Shouldn't those fix spam for good

Not quite sure I follow? Unless you mean "only allow emails to specific email addresses you've noted down", in which case, yeah, that works but also means a lot of admin when you want to use a new one (plus there's 10+ other people who use my servers for email, not just me.)

> unless you publish your address globally?

At least one of my email addresses has been published globally since ~1995. Others since ~2000.

By lot of admin you mean that it's not automated? That's sad indeed.

> By lot of admin you mean that it's not automated? That's sad indeed.

How would I automate "[someone with an account] just entered abc-xyz@domain into a web form to subscribe to something, add that to the valid alias file"?

As I understand, spam filters like postscreen are designed as reverse proxies or relay servers. IIRC spamgourmet supports your scenario.

> IIRC spamgourmet supports your scenario.

Not really - that's a limited use alias. Doesn't really work for when you want to keep getting email from places. Also if I'm reading that correctly, spammers can just send mail to <randomword>.20.<knownaccount>@spamgourmet.com and you can't protect against those 20 spams.

They can, but reportedly they don't, see FAQ. Also unlimited emails and locked senders are described in FAQ.

You are not a valuable target to spammers. At some point even storing the spam letters makes no financial sense.

I think that gray listing is a great idea to filter out spam. Here's how it works: the very first time an email server get an email from a new IP it answers with a "temporary unavaiable" error. According to specs, a server should retry to deliver the email after a while, so a legitimate server will retry and it's IP will be put in a "withelist" (here in quotes because you can still do further processing of the emails to determine if it's good or bad). But a spammer will most likely not retry to send it, as their goal is to quickly send a big amount of mails. I didn't try this personally as I'm not hosting my own emails (I tried but gave up soon), but I heard it works very well, no third party blocklist needed.

> I heard [gray listing] works very well

Mostly. But then you get the "click the link in the email within 10 minutes" problem. There's also a non-zero number of "our mail didn't get through first attempt, oh well, give up" people. From running GL on my servers over a couple of years, it mildly cut down spam (on top of blocklists and fail2ban) but I'm now wavering over whether it's worth the hassle.

It cut spam tremendeously for me, but that's on an email address I've used and published openly since early 2000s.

Still, I've given up on it since plenty of email senders are not standard-abiding (they fail to retry), and I've kept losing email. I only caved in in the last 12 months after 15+ years of doing graylisting.

The trade off is that even legitimate mail that you are actively expecting (like an account confirmation or password reset mail) will be delayed by however long the sending server's retry interval is.

You also either need to apply the greylisting to some larger IP range (rspamd e.g. apparently uses /19 by default for IPv4) or otherwise specially handle some of the bigger mail providers, because some of them rotate through their servers between retries, so you could be in for a quite a long wait if you do per-individual-IP greylisting.

The biggest culprit I noticed this with was Amazon SES – a former mail provider of mine used per-individual-IP, non-configurable greylisting, and any mail sent through Amazon (which isn't just Amazon itself – quite a few companies are using Amazon SES for transactional mail and suchlike) would consequently almost always arrive several hours late (however randomly long it would take Amazon to finally re-use an IP during a subsequent retry attempt).

Even more infuriating, my mail provider's support would then claim that it wasn't their fault and they didn't know anything about any supposed greylisting.

You just solved a random mystery I was having with mail forwarding from a single particular provider.

I have greylisting enabled for years. But recently some Russian spamers were able to circumvent those by using compliant SMTP servers. They even support proper SPF, DKIM, DMARC and all that stuff what you can think of.

That is the reason why I switched on some external block lists into the mix.

I wonder if anyone compared the effectiveness of postscreen vs greylisting. Can't find any

Both rely on filtering out non compliant senders, but postscreen's filtering might be less disruptive. Are there spammers out there who cannot pass a graylist but can pass postscreen ?

What we found 20+ years ago when we started to provide a very poor service to attempted spam delivery was that the spammers realised our servers were very 'slow' and stopped the attempts. We used to get 1000s of new IPV4 tuples / day, now we get < 10. If you put the inbound SMTP sessions to sleep this is wrecking the spammers bus. model, they need to deliver 1,000,000,000+ / day. Don't u fell sorry 4 them. The only time we don't tuck them in 4 a snooze is if they are in a 3rd party BL !!!

This is pure victim-blaming. The problem is that bulletproof hosting exists, not that people are sick of spam.

I don't follow. I am pointing out that using the same kinds of lists that the big providers use will result in smaller providers being arbitrarily blocked with little recourse. Do you disagree with that assessment?

First, yes. They (the small providers) have recourse, it’s just annoying. They voluntarily signed up for that annoyance because of their ideology.

But much more importantly, that question is orthogonal to my argument. Using blocklists is a good way to cut down on spam, the fact that it might block some trivial percentage of people who for ideological reasons might or might not be on those lists isn’t the receivers problem.

You want this to not be the case? Contribute meaningfully to solving the problem in a more effective way. Don’t blame the people who just don’t want the spam.

> ideological reasons might or might not be on those lists

What's the ideological reasons here?

Apparently people who want to self host email instead of paying some big tech

The small providers typically don't have recourse if they approach a blocklist maintainer and ask to be taken off a list. That "trivial percentage of people" make up a substantial portion of service providers (for every Google there are many <100 user mail servers), which is what this thread was about.

I pointed out that adopting blocklists just makes it harder to operate a mail server as a small provider, and nobody in this thread appears to disagree with that assessment. They instead seem to take issue with the tone of the message.

>The small providers typically don't have recourse if they approach a blocklist maintainer and ask to be taken off a list. [...] I pointed out that adopting blocklists just makes it harder to operate a mail server as a small provider, and nobody in this thread appears to disagree with that assessment. They instead seem to take issue with the tone of the message.

I don't think it's the tone. While the inability for senders to get off blocklists can be true, you're still not addressing why the cost to the sender to not be blocked should have higher priority than the cost to the receiver to avoid blocklists to make email admin more of a burden.

The gp you first replied to (PinguTS) is running a personal mailserver and resorted to using blocklists because reducing spam -- at the cost of some legit people not being able to send email to him -- is a tradeoff he's willing to make. You haven't convinced every user running mailservers that they should increase their spam burden because some small providers can't get off blocklists.

As another example in another communication channel... Here's a similar "blocklist" for cell phones that some users take advantage of: https://about.att.com/pages/cyberaware/ae/cp

Are "legitimate" phone numbers getting caught up in that block filter?!? Of course. But phone owners are trying to stop spam telemarketing calls. Telling them that some phone users are incorrectly on AT&T's list isn't going to convince cell users to quit using the block filter. They're willing to live with a few legit callers getting blocked.

> Even as I don't use DKIM or DMARC.

I wouldn’t expect sending email to be the problem. But I’d be surprised if it’s delivered.

One of SPF or DKIM is pretty much mandatory, and SPF is far easy to maintain.

Reply on news.ycombinator.com