Exercising Software Freedom in the Global Email System | Hacker News Reader

Exercising Software Freedom in the Global Email System

sfconservancy.org

64 pointspmoriarty19 comments

Very thought-provoking.

Summary: Ancient protocol (SMTP) gets overwhelmed by modern requirements (spam prevention). Instead of updating the protocol (it's too entrenched), we hack around it (blacklists). But the hack is outside the standard, and it's more social than technical, so our software doesn't address it.

Could we standardize the blacklists so that we can address them with open/free software? (Assuming we don't replace SMTP itself.) Is there general interest in doing so (or conversely, is it against the interests of current service providers)?

Public blacklists are fairly standard, based on DNS. Removals less so but tracking down the right form is usually simple.

Large providers, however, either aren't using these or augment them with in-house, non-standard, and undocumented lists and mechanisms.

Someone has to maintain the blacklist, I don't think it matters whether they do so using free software or not.

The other issue, not raised by this article (probably because the author hasn't hit that problem yet) is that some providers like gmail seem to flag things as spam just because the sending server is unknown. At least, I have mail being put into spam folders for gmail users even though my servers are on no blacklists. And no, I don't make a habit of sending enticing offers for "FREE V1agr4!!1"

What I get out of this is that Microsoft Outlook is a bad webmail service that will mark things as spam even though there is no evidence they are spam.

I have recently seen more or less the same thing implied about Gmail.

So the actual issue seems to be that free webmail services are not that great. Historically free services tend to get worse. Perhaps it is just a coincidence that Outlook and Gmail are degrading in the same way at the same time.

The problem is centralisation. As most email users move to the 3 or 4 large providers, the incentives for those providers to care about email delivery to the smaller players is reduced.

Having run my own mail server for about 20 years now, I can absolutely confirm that this is a big problem and getting worse all the time.

Gmail is in my experience the worst. And my IP is on no blacklists as far as I can tell. Mail just gets silently dropped, and I only find out about it when my sister gets confused when I ask her why she hasn't responded to my mail.

>The most notable change in the email world is the influx of massive amounts of spam, which has been used as an excuse to implement another disturbing change. [...] These problems do reduce email software freedom by network effects.

This essay is fundamentally flawed.

Because tech people like us are intimately familiar with computer topics such as "hardware", "software", "TCPIP", "RFC", etc, we often frame phenomena in terms of what we know. (E.g. the writer really knows how to set up an email server from scratch and has done so since 1993.) In this case, the writer has isolated a particular component as "free" and then worked his thinking outward from that. The "free-ness" he's fixated on has distorted the thinking and is causing all other costs to be ignored.

  - The Unix sendmail,postmail,etc programs are free.
  - Mozilla Thunderbird, Roundcube, etc are free.
  - SMTP specification has no license costs and is free.
  - They are all free in both meanings as in libre ("free speech") and cost ("free beer").

However,

  - Trust is not free.
  - Reputation is not free.
  - Diskspace is not free.
  - Bandwidth is not free.
  - Time and attention for eyeballs to look at unwanted email is not free.
  - all the above contribute to umbrella effort of "fighting spam"

To understand why thinking outward from the phrase "softare freedom" is a flawed framework, replace it with "English Language freedom".

We can all use the ~200,000 words[1] of the English Language without paying EnglishSoft Inc a monthly subscription fee.

Despite my EnglishLanguage freedom, it is being blocked by other agents and gatekeepers. I should have the freedom to communicate my English to anybody including actors, rich people, and high ranking officials.

For example, all the following middlemen are trampling on my freedom:

  - The agents for Tom Cruise and Jennifer Aniston who screen everyone's calls.  
  - The secretary who blocks my communication with Warren Buffett.
  - The White House Chief of Staff and other gatekeepers to the President.

In another related example, we have the global phone network. In some sense, we collectively have a "phone dialing freedom". However, I wish there were big companies that I could pay to screen every call on my behalf. Many people would want them to keep centralized blacklists on annoying telemarketers, political pollsters, etc. For example, I'd prefer Verizon/AT&T maintained the blacklists instead of me keeping adhoc blocklists on my iPhone for every unwanted call. I don't want my phone to ring at all for such calls. I also don't want a spam voicemail that I have to later delete. If I use such a service, did I trample on your "phone dialing freedom"? I certainly did! I think many of us would like the "freedom" to do that!

Therefore, focusing on "English Language freedom", "phone dialing freedom", or "software freedom" will make one blind to the bigger picture of why blocking&filtering manifests itself as an emergent property of any communications network. The fact that it's also crazy hard to communicate credibility to get off those blacklists and honest people like the author suffered is unavoidable. Otherwise, spammers could also trivially remove themselves from blacklists.

[1]http://www.oxforddictionaries.com/us/words/how-many-words-ar...

I'd be interested in guides related to setting up a RFC-compliant email server with free software. Any suggestions?

Basically all popular smtp servers will get you an RFC-compliant setup. Still doesn't help you get your mail delivered to parties like google and microsoft.

First pointer would be to make sure you have your DNS setup correctly including reverse-dns. But the problem is, as described in the article, that too many commercial parties have their own (undocumented/unpublished) blacklists and aren't too forthcoming with removing entries from them.

If you have an IP with a good (spam) reputation... hold onto it, it's getting more and more important.

I find it significant that the party you are delivering your emails to is not the recipient, but instead google and microsoft who then decide if the message will reach its intended destination.

https://yunohost.org/

https://yunohost.org/#/whatsyunohost

YunoHost is a server operating system aiming to make self-hosting accessible to everyone. It is based on Debian GNU/Linux and is fully compatible with it.

Basically YunoHost automatically installs and configures some services around LDAP, and provides tools to administrate them.

It can thus be considered as a distribution, including the following software:

    Nginx: a web server
    Postfix: an SMTP e-mail server
    Dovecot: an IMAP and a POP3 e-mail server
    Amavis: an antispam
    Metronome: an XMPP server
    OpenLDAP
    Bind: a DNS server
    SSOwat: a Single Sign On (SSO) web authentication system
    A backup system (not yet implemeted)

This is same thing I said on this thread: https://news.ycombinator.com/item?id=10486062

Your own Debian Mail Server (part II): how to prove you are not a spammer

https://news.ycombinator.com/item?id=10486380

> Any suggestions?

bkuhn's upcoming guide on the matter which he just announced, I hope.

Not related to this article directly, but what is the current perception of the term "Software Freedom"? I know that FSF equals it to GPL, which grants certain freedoms to the user of the software, but if such a user is actually a developer, one freedom is quickly taken away - one can't use the software as one wish, but needs to also license his work as GPL.

Of course it looks like I am trolling here, but I really try to understand what is so great about GPL. I noticed large companies like Google or FB are releasing their open source projects under Apache license, which also grants the same freedoms like GPL, but also offers more - truly use the software as one wish. And for some works, like for example FB's folly C++ library, it is really a high quality cutting-edge stuff, and anyone can still contribute like in GPL world.

This is such an old BSD vs GPL flame. And it's irrelevant here. You will run into the same issues when using a high quality, cutting-edge, BSD-licensed MTA.

At the same time, however, there are plenty of companies that use software released under a permissive license without contributing back -- they release none of their improvements, they don't contribute financially and so on. GPL adepts hope that it will force these "malicious" companies to comply, while doing no harm to the ones that would contribute anyway. Or, in their own jargon, that the freedom (to reuse that software for any purpose, including commercial ones) which made this possible is preserved.

The results have been... mixed, as far as I know. Permissive licenses do allow many commercial players to "abuse" the system -- and make no mistake about it, especially companies like Google or FB are entirely disinterested in charity. "For the community" is a convenient PR stunt, but real life has consistently shown that every large company has enough ulterior motives to make community service an accident, not an intent.

On the other hand, GPL (rightfully) scares companies away. Even fair players shy away because of the complexity and difficulty of the license, and developers are sometimes put off by the amount of politics involved. These things discourage legitimate contributors.

This discussion has been done so many times. There are advantages and disadvantages to all the major software licences. We don't need another 200 comment flame war about the GPL.

> one freedom is quickly taken away - one can't use the software as one wish, but needs to also license his work as GPL

Licensing software to users is not the same as using it for oneself; it's not use at all. There is no freedom to license someone else's software; there never has been a freedom to profit off of someone else's labour (there is, of course, a freedom to try to do so, by paying someone to work and then selling that person's work for hire).

> I really try to understand what is so great about GPL

It creates a community of users and software, where user-developers know that their contributions will be freely available to other users, and will not be locked up.

In principle, there's much to admire about the BSD license. But, as we see with so many proprietary devices and software built on BSD-licensed software, it's simply all too likely that good code will be locked up inside proprietary cages.

> it's simply all too likely that good code will be locked up inside proprietary cages.

...and that bad changes to good code get hidden and are now unfixable.

> I know that FSF equals it to GPL,

No. It makes me sad that people still believe this. :-( Free software does not mean GPL. The FSF is quite explicit about what licenses it considers free, and they coincide almost exactly with the licenses that OSI declares open source.

As many has said, this is a very old flame war and unnecessary in this discussion. There is plenty of articles and discussion forums that can provide a suitable environment for this.

If you want to understand the philosophy about freedom, just take a glancing look at any book that talks about it or liberty and you will find plenty of answer to your questions. John Locke for example put freedom into two camps, nature and people, where freedom of people means to be under no restraint apart from standing rules to live by that are common to everyone. Freedom of people can thus be limited by a law, a license, a restriction, and people are still free so long it effects everyone in a equal (common) way. It is thus not freedom of people to use something as one wish without also follow common laws.

A few hundred years later, Isaiah Berlin formally framed the concept of negative liberty as the freedom to not be interfered by other persons. One could thus describe any license which for example prevents patents from interfering with software development to be licenses that preserve software developers freedom to develop software.

Those a just small samples of the philosophy and of course others has written their views on the subject, and I recommend reading those if you are really trying to understand Software Freedom and GPL.

Reply on news.ycombinator.com