Hacker News Reader

Comment by ttul | Hacker News Reader

A little-known fact: WHOIS is extensively relied on by spam fighters like Spamhaus to do their good work, which collectively saves all of us from an enormous tidal wave of spam that would otherwise consume vast resources. Internally, the anti-spam and more generally the anti-abuse community builds a huge and mostly real-time cross-referenced database of information about domains and IP addresses. This database would be impossible to build without WHOIS.

For example, if one domain shares the same contact email address as another, then the domains are related somehow. Doing some data mining on a variety of signals which are apparent in the WHOIS records can help to cluster related domains to help anti-abuse researchers find newly problematic domains by following the trail through WHOIS.

I'm not sure how researchers will do their job effectively without WHOIS. This development is truly a disaster for anti-abuse.

View on HN

Never mind the WHOIS, those other anti-spam databases may be GDPR-violating, as well. Did that spammer affirmatively opt-in to having his origin addresses/domains/IPs and response phone-numbers/etc tracked in your anti-spam data? Probably not!

That doesn't seem to be personal information.

I believe IP addresses are considered "PII" under GDPR, since they can be used to identify an individual.

https://gdpr-info.eu/art-4-gdpr/

https://eugdprcompliant.com/personal-data/

"PII" only seems to exist in US law, GDPR has "personal data".

I am far from an expert an GDPR, but it doesn't seem to be so clear cut. Even if IP addresses in this context are considered personal data, there may be "legitimate interest" in processing them for blacklists, e.g. https://gdpr-info.eu/recitals/no-49/ could apply. I am confident a workable solution for spam blacklists will be found.

I have the impression that a lot of the fear around GDPR is unfounded if one uses a reasonable and restrictive approach of processing and storing personal data.

Yes, IPs and any other information can be kept if there is a legitimate interest. For example if another regulation requires you to keep full information for AML or tax purposes, you can't immediately comply with a right-to-be-forgotten request to delete all the data you hold.

It's still personal information though (which was my original point), and so you still need to comply with GDPR by minimizing usage, not sharing it to processors without permission, having a procedure for telling users what data you hold on them, etc. And I think you'd have a harder time claiming that the other stuff is required too, specifically the addresses and phone numbers. You can do spam detection without that information, even if it would be less effective.

The problem I see with GDPR is just that we won't know precisely where the boundaries are until there's some case law to set precedent. It may prove to be easy to comply with, or it may prove to have some sharp edges that are expensive to comply with; we really can't tell.

The same WHOIS is used by spammers to send annoying emails do domain owners

Domain owners are a tiny, tiny fraction of the population spammers are attempting to reach.

> WHOIS is extensively relied on by spam fighters like Spamhaus

Does anyone of importance still use those? Google and other major email hubs have long switched to AB testing and building user profiles as their primary filtering tools. They want to gather that data to improve efficiency of their targeted advertising, so I trust them to be good at it.

Smaller players might not have resources for that, but how do those opaque third-party blocklists help them? In the best case, those "anti-spam communities" do nothing. In the worst case, they act as data-harvesters, potentially leaking information to (lol) _spammers_. Why should we care about their future?

How does AB testing and user profiles have anything at all to do with detecting when incoming email is spam?

This post explains it: https://news.ycombinator.com/item?id=12282894

TL;DR: modern email providers don't care if you are in blacklists. If your IP/domain does not have established reputation, they will drop half of your email in spam folder. If users whitelist it or reply to it, your reputation automatically improves.

If your send too much email or your receivers blacklist you (delete without reading or manually move your email to spam), your reputation takes nose dive. Some providers (for example, Yandex) openly describe that logic in their FAQs.

Intersting.

However, as someone who uses SpamAssassin (via FastMail), spam clearinghouses like Spamhaus are still very important. And as long as we want to avoid centralizing all email in the hands of a few massive providers, they will continue to be important.

And for mail senders, SpamAssassin, especially in a comparatively large deployment like FastMail’s, is super useful because it actually tells you why a message was classified as spam, and it’s mostly actionable. SpamAssassin’s rules won’t be the same as what Gmail uses, but there will be strong similarities in many of the rules, and so the sorts of actions that may be necessary to get your SpamAssassin-assigned X-Spam-score down are likely to help on providers like Gmail too.

(I just wish there was better documentation of what all the rules mean, and how to satisfy them.)

I don't get it.

What the current system does: a private person who registers multiple domains with his own name and then proceeds to spam email users, that person can blacklisted effectively because his persona information can be fetched and matched.

A spam virus is not the problem here, because personal data does not reliably connect infected domains.

And spam companies are not problem here, because GDPR is not concerned with companies.

The person who uses several domains to spam needs to be somewhat aware that he is spamming. But he also needs to be incredibly idiotic to connect his spam domains with his personal info.

Even if WHOIS going down is short term tragedy to anti-abuse, GDPR does not seem to prevent building a replacement that works well enough.

Reply on news.ycombinator.com