Zed Shaw's Son of Sam Email Server: Write Email Apps in a Modern Style | Hacker News Reader

Zed Shaw's Son of Sam Email Server: Write Email Apps in a Modern Style

zedshaw.com

83 pointsmqt37 comments

re: performance, any smtp mail server worth its salt will be limited by I/O write performance (email should be sync'd to stable storage before signalling acceptance). CPU spent pushing data over pipes/unix sockets won't register. I don't see any measurements in there, so it sounds like "X Windows is slow because it has to talk over a socket" analysis.

"The short version: Classes and methods beats pipes and processes."

Well, it depends. For latency on when processing a single large email, yes, perhaps. One-pass processing will give you that (although I think he skipped over how SoS manages to hand off incoming email to SpamAssassin without "passing it over a pipe to a perl process")? Surely we are trying to compare like with like, right?

But mail processing is a parallel activity. Reducing latency on a single message is not the same thing as improving throughput. What do we find out about that?

"But, sendmail and postfix are still way better at huge scale bulk, so it relays to them."

Ah, OK. So this is a boutique solution, for personalised handling of small-site stuff. Guess what? We already have that, since (as noted by the spamassassin example) you can set up the big solutions to hand off a message to you to process in the language of your choice.

Sorry for -ve rant. I didn't see any -ve comments in this section and wanted a little critical balance. Nice software is always nice, and you don't have to have a reason to want to write it.

But I don't like unjustified justifications. If you're going to claim better performance, give numbers.

What is "-ve" ?

Negative, sorry for abbreviation.

Now all he needs are IMAP and POP3 implementations and we might finally have a mail stack that mere mortals are capable of setting up without screaming profanities.

I found Dovecot pretty easy to install, but setting up spam filtering (especially SMTP-time) is a lot of work.

Any of the puzzle pieces individually aren't bad, it's putting them all together then adding additional pipes to stuff like (as you mention) SPAM filtering, etc. that make it such a huge pain in the ass. In an ideal world, there would be some package available where you simply specify the credentials for the machines you want to use for each piece (smtp, pop3, imap, secure pop3/imap interface, etc.) and it will setup everything for you.

In an ideal world.

You reckon 500 lines of Python, whipped up on a whim, provides everything you need in an MTA, other than IMAP and POP3? Really?

Add five years of heavy development, and it'll probably be a suitable replacement for Postfix or qmail or Sendmail.

Besides, what's hard about Postfix? It usually only requires two or three configuration file changes to get mail processing working.

Does anyone else have trouble reading Zed's blog because of the high contrast white-on-black text theme? I have to look away after around a minute, and even then I have ghosts burned into my retinas.

In Firefox: View -> Page Style -> No Style.

r.e. title image, does anyone else think the chick on the left looks like she has a dick?

Sounds Freudian, dude...

http://en.wikipedia.org/wiki/Freudian_slip

That, but used differently.

I wonder why he chose Mako as a template system as opposed to Jinja (django style). Looking at the Mako syntax.. it just doesn't make any sense.

Django templates are designed to be designer (non-programmer) friendly, becase of its CMS/newspaper origins.

Mako's goal is to create a powerful, programmer friendly and FAST templating language .

A lot many times having code in templates is a faster way to develop, especially if the presentation layer is mostly intended to be used by a programmer.

Also, you can write mako code which has good separation of concerns.

How is : <% for r in rows %> ${r.name} <% endfor %>

less readable than :

{% for r in rows %} {{r.name}} {% endfor %}

jinja2 is as fast as mako. mako is popular, though, so I don't begrudge anyone choosing it as a default for a project.

jinja2 also lets you easily change your delimiters to make it suit your style, which is nice.

As it happens, I advised him on template choice; I recommended both Jinja2 and Mako, and he ended up choosing Mako because it supports embedded arbitrary Python blocks, and Jinja2 does not.

Interesting. I wrote something very much like this for internal use at my company. Basically we have a lot of legacy alerts and messages and reports that are "dumb" (i.e. if a cron job runs once a minute to detect problems within one minute it will keep on mailing until it's fixed because it has no state) so the new mail system sits in front of it all and says, hmm, saw that a minute ago, won't bother the humans with it again, they're probably busy actually fixing it.

It's not, and never will be, a general purpose mail server - old-fashioned SMTP servers have their flaws but it's as good as it gets if you want mail that can reach anyone, anywhere.

Does Twitter have message-ids like email?

I ask because email has the property that all replies are linked to the original message, creating a dependency tree of ID's. Any system that doesn't create a unique message-id is a step back from email.

Message-Id is actually optional in email (recently brought up when trying to make a threaded email view: http://www.mail-archive.com/couchdb-user@incubator.apache.or... )

Hi, thanks for pointing this out. I know that some web mail systems or Outlook mailboxes seem not to have the original ID, and one has to be generated in some random way (that breaks the links).

Twitter messages have unique URLs (permalinks). It doesn't exactly have replies, though; you can send a message @someone (public, but intended especially for the given person) or send a private direct message, but there's no standard way to indicate which message you're replying to.

This is an intentional step back from email. Twitter is a bad place to have long back-and-forth conversations by design.

The Twitter timeline knows the specific message a reply is connected to.

... but is actually just inferring, and frequently gets it wrong.

With in_reply_to_status_id and proper client support, that should go away, hopefully.

Twitter messages have unique IDs as well as an optional "message replied to id" (in_reply_to_status_id). The latter defaults to the last message from userX if you send a message with the @userX convention.

Thanks for confirming about these message-ids with Twitter.

Tis the season to write SMTP implementations.

http://undeadly.org/cgi?action=article&sid=2008111208464...

http://smtpd.develooper.com/

And FWIW, qpsmtpd handles a ton of mail with no apparent problems.

With that in mind, scaling a mail server is about speeding up disk access, not implementing it in a "fast" language. I'm sure Zed's thing can do just as well.

Is it okay to give a technology the name of a serial killer?

If you are Zed Shaw, yes.

I think he chooses names just to offend the kind of people who would find those names inappropriate in a business setting.

Bell Labs named three of their operating systems after castrated men, a notoriously bad movie, and Hell itself. Names that make marketers cringe have a long tradition.

And let us not forget SATAN.

Which was written by Wietse Venema, who went on to write the finest general purpose SMTP server in existence. But, he chose a more suit-friendly name (Postfix) for the mail server.

Don't forget Dan Farmer also worked on SATAN. And FWIW the name history on Postfix was: VMailer, then (in parallel) IBM Secure Mailer (forgotten) and Postfix (alive). Wietse's previous work (tcp_wrappers?) didn't have particularly suit-unfriendly names either. Maybe it was Dan who suggested SATAN?

Reply on news.ycombinator.com