Comment by csnover | Hacker News Reader

Comment by csnover | Hacker News Reader

Parent Full thread

I guess my question is can you please fix your braindead blacklisting?

Several times per year—I can practically guarantee it’ll happen sometime in December, and indeed had to deal with this just five days ago—I end up with a bunch of users whose email notifications stop working because Microsoft have started blocking the entire netrange where my server lives. I don’t have control over other Linode customers, guys! I even wrote extra code to stop sending mail to addresses that start bouncing specifically to avoid blacklisting, so after MS finally processes a blacklist mitigation request, someone also has to go in and re-enable those accounts.

SPF, DKIM, DMARC are all configured; I’ve sent from the same IP address for about a decade; I’ve not once received an email abuse report; mail volume is low (most days, volume does not reach the minimum threshold for SNDS to report data[0]). I’ve never had any other mail provider blacklist my server. SNDS always says everything is OK as I am S3150s. What is even the purpose of SNDS at this point when it lies about what is going on?

[0] P.S. The janky SNDS calendar widget resets the month to the current month every time you click on a date, even if the date being viewed is in a previous month. I don’t have any hope that anyone will ever touch SNDS code again since it was clearly designed in the early 2000s and the copyright on the site is now ten years old, but this is a pretty silly bug.

My guess is that the effectiveness issue isn’t actually due to SNDS and is probably related to sender reputation having famously high false positive rates. I read a paper a while back which introduced a different algorithm with tighter bounds on regret, I didn’t really understand it tbh, but I can implement it behind a flight and run a data study to see if it works better. The problem is that most graph based stuff doesn’t scale super well because of something-something complexity classes. I think the lady who architected it 5 years ago didn’t do a great job and there’s a bunch of arbitrary config stuff which was put as a placeholder and then became enshrined in stone… but the guy maintaining it rn is really smart so I’ll have him review my half-assed PR when he’s back next week (and idk how long it’ll take to finish the other half of it, shit never ships around here).

About the calendar widget thing… man am I glad I our team doesn’t own that. No one ever touches legacy stuff cause they’re afraid it’ll break or no one will update but the trick is to file it as an accessibility bug since that gets someone to actually prioritize it since it shows up in reports that the execs read. But dude good luck getting that off the backlog, the one engineer we have who is good at UX stuff (i.e, can code with both quality and velocity instead of just one) has her hands full as is.

Whatever the problem is, all I know is that last year Linode said they tried and failed to get Microsoft to actually fix the problem[0], apparently despite assurances and multiple requests for a root cause analysis. Everyone else seems to have figured out how to not be overrun by spam and also not block entire netranges, so I’d say it is well past time for Microsoft to figure out how to do that too.

Thanks and good luck!

[0] https://www.linode.com/community/questions/22287/550-57511-a...

> I read a paper a while back which introduced a different algorithm with tighter bounds on regret

Got a link or cite for that paper? It sounds interesting.

https://arxiv.org/abs/1411.1127

> I don’t have control over other Linode customers, guys!

You do have control over being a Linode customer though. If Linode isn't doing enough to prevent abuse, they deserve to be blocked.

Asking infrastructure providers to police email content is a very invasive thing to want. I don't think I agree with that.

Realistically, what can they do here? Make servers unaffordable to discourage abuse? Give most servers "Internet*" access where some ports are missing?

> Give most servers "Internet*" access where some ports are missing?

Disallow SMTP traffic unless an account has a certain reputation or verified identity related to it?

I mean, they don't have to do that, and I would agree the government shouldn't force it to happen. But if someone is constantly causing you problems you shouldn't be required to deal with their shit. If you don't want to behave, expect consequences from everyone else in society.

If every time my friends invited me over I brought over another random person that smears feces all over the walls and pees in the corner I probably won't get invited over very often. Linode (and other cheap VPS hosts) are that person constantly enabling abusive people and subjecting them to others.

Personally inviting someone to your party? Surely the analogy to linode is something like an apartment building owner. You wouldn't ban your established friend just because you keep having problems with nearby tenants.

Oh man I think around 2 years ago there was a 3x spike in Europe outbound spam and the fraud team had to disable like 200k+ tenants from some shady cloud VPS. We didn’t have a long term plan for the abuse back then besides playing whack-a-mole, and if we have one now, I haven’t heard of it.

Dumb question, but wtf is the solution even? I’m confused about what you expect us to do. I haven’t thought about the problem much so I might be missing some obvious Pareto improvement.

Did I accidentally give the impression I have a particularly good solution to spam in mind? I don't.

My main suggestion here is just that an established reputation for a specific IP/domain should override the reputation of the IP block it's inside of.

You didn’t make any mistakes encoding, I just screwed up my decoding, it happens :D

Thanks for clarifying, ok, hmm… that seems hard to do if you can check the IP block by using a subnet mask but the specific IP isn’t resolved until later in mailflow. It might not actually work like that in… ProtocolFilterHub? I always get this mixed up, wait… I think this might be something that we are already working on. And have been working on for a while, wow. Looks hairy. It’s stuck since the guy working on it transferred to another team, and no one picked it up, but some PM noticed before I did and put it up for vote in semester planning. Always creepy to see engineers get referred to as “resources”.

But that's how the world works right now: every provider has acceptable use policy, and not just for emails. Not necessarily because they care, but because they are beholden to an AUP, from their upstream or peers. Which makes it viral: if they won't hold AUP, they'd get cut off, and there's very little use in an internet service provider without connectivity.

Linode respond swiftly to abuse reports[0], block outgoing SMTP by default, and prevent so many people from even registering for services that it is the #1 question people ask on their IRC channel. What more should they do? What is “enough”?

90% of the spam that I receive from a DMARC-validated sender comes from Google; should every Gmail user be punished because Google aren’t “doing enough”?

[0] Linode twice threatened to shut off services within 24 hours due to some vigilante scanning the internet with a broken virus scanner and automatically sending reports: https://virtuallyfun.com/2014/04/23/dumbass-of-the-year-awar... (n.b. this is not my site)

> Linode respond swiftly to abuse reports[0], block outgoing SMTP by default

One instance of them supposedly responding quickly to an email abuse report isn't showing they're consistently responsive to abuse reports. I don't know if they are or are not. I don't even know that this blog post even refers to Linode, they're not mentioned once.

And its not true they always block outgoing SMTP by default. Loads of old accounts do not have SMTP blocked. New accounts since 2019 sometimes have it blocked, but given the last few times I've made an account and didn't have any blocks it doesn't seem that often. Maybe I just got lucky though.

And don't get me wrong here, I'm not intentionally singling out Linode here. There's loads of cheap VPS providers that enable this kind of abuse. They're not necessarily better or worse in this regard to many others.

> 90% of the spam that I receive from a DMARC-validated sender comes from Google; should every Gmail user be punished because Google aren’t “doing enough”?

Yes. Just like those telephone companies originating most of the spam phone calls should get disconnected. If they're going to enable abusers, they should get cut off.

Reply on news.ycombinator.com