Hacker News Reader

Comment by bawolff | Hacker News Reader

I largely agree, although i think the author misses on some of the examples.

Reddit is not an example of being bad due to this affect, but an example of being good because of it. Reddit is not one huge platform but a combination of thousands of subreddits independently moderated. That is how reddit remains good, because small subreddits have small scale.

Running your own email server isn't a counterexample, but is legit terrible because in essence you aren't moderating a small community, you are moderating the whole world, but just your view of it, since anyone can email anyone. That is the worst because it is the worst of both worlds - no budget from being big scale but still all the problems of being large scale.

View on HN

My experience running my email server is exactly what danluu says. Receiving emails and classifying spam is not a problem at all, but my outgoing mail is often blocked. It's not that I struggle to moderate the whole world, but the whole world is moderating me.

> It's not that I struggle to moderate the whole world, but the whole world is moderating me.

Wouldn't that be the "best case" for email because of the centralized idea? In reality it's a few huge companies that matter and if you end up on one of their lists because of some bad IP, missing header etc. you are a bit out of luck.

It is in fact very instructive, given that email is an open federated protocol but it doesn’t matter because the industry has centralized.

We now have several new social networks based on the ideas of open protocols and federation. They are not yet large enough to attract too much attention but we’ll see if the story rhymes.

It’s not “the whole world” moderating you, it’s more likely than not specifically Google. In my age and peer group, personal usage of GMail is 98%+, professional usage of GMail is 50%+. Not delivering to GMail is not an option, especially so if the email is not just for a social purpose but, for example, for finding a job.

I ran my own email for close to 15 years, most of that time on the same IP address and the same old domain. I could never get GMail to reliably accept my mail, and of course there is no support to find out why. I set up and tested their documented best practices, and had no issues corresponding with a lot of people.

Spam is not an issue in practice. Two basic filters drop 95% of spam without scanning the content, and an open source reputation+content engine drops the rest to the tune of maybe 1 spam a day. The two annoying parts are dealing with exceptions, often large companies misconfiguring their outgoing email; and the open-source nature of the filtering engine’s ruleset, meaning it occasionally completely breaks for a short while.

Worse, spammers are better at their 'professional job' (getting mail past other people's spam filters and inane preferences) than your tiny mail server or small group of employees who don't understand how to not look spammy. Like not having annoying email 'signatures' and other anti-patterns commonly found in corporations.

> Like not having annoying email 'signatures'

What?

I used to work IT helpdesk and once spent weeks trying to find out why my company’s emails were getting bounced all of the sudden. We had just switched to Microsoft’s hosted exchange (office 365) so we figured that must be it. I escalated the issue until I got some very technical folks on the phone, but all they could say was that everything looked 100% fine and that something we were sending was tripping the spam filters. I thought about it and realized we recently changed our email signatures to include a bit.ly shortened link. I guess spammers also use that to obfuscate their links so we were indeed sending out sketchy email

I mean that one was pretty obvious. Why on earth send a bit.ly link in every email? You have a-hrefs in email, so the link doesn’t even need to be shortened.

> I got some very technical folks on the phone, but all they could say was that everything looked 100% fine and that something we were sending was tripping the spam filters

This “we won’t tell you what triggered it because it helps the spammers” is… questionable, and it feels like security by obscurity. Yes, it may help the spammers a little but not possibly enough to warrant the black hole of debugging legit issues.

The right way to fight spammers isn’t about arms racing with hyper-advanced content filtering. Instead, we need to make it uneconomical.

The bit.ly link was to track how many people clicked on it, for marketing purposes I guess.

As for the support team on our email provider’s side, they didn’t know what was tripping it. We simply got to a point where we ruled everything else out. The only thing left was the content of the email being legitimately spammy, somehow.

Our email provider wasn’t the one that was filtering our emails, at any rate, rather it was the recipient email systems.

Things like additional inline images, hyperlinks to a bunch of places, etc non-text data. It's probably only a couple fractions of a point more spam score per message, but spammers and typical employee end user communication can look annoyingly similar.

> you aren't moderating a small community, you are moderating the whole world

Sure, but you are still moderating for a small community. One person's spam is another's ham, so it's much more difficult for Gmail to be as accurate as my spam filter (which only needs to learn from a couple hundred people's preferences) because it will inevitably overfit.

Reply on news.ycombinator.com