Comment by Animats | Hacker News Reader

Comment by Animats | Hacker News Reader

It's hard to make money in that space, yes.

I've considered merging mail and messaging, so that email protocols are used to do the things people do with instant messaging, Telegram and such. The first step is to get away from queuing mail.

A first step is a new kind of mail forwarder, one for machines that don't have local mailboxes (which is most non-mail servers). It acts as an SMTP server and client. When an email comes in, and the sender reaches the end of the DATA section, the forwarder checks its forwarding list to see where this mail is supposed to go. It opens a SMTP connection to the destination and sends the email. The incoming SMTP connection is held open while this is in progress. Any problems are reported back to the sender immediately with an SMTP status code. There are no bounce emails, no queuing, and no possibility of joe-jobs. All mail is handled immediately or rejected immediately. (This only applies to single-address messages, and only to "To" addresses. Everything else is bulk.)

The next step is an IMAP server which has the same immediate orientation. When it gets an incoming email, it sends a push notification to any connected clients, while holding the incoming connection open. If at least one client is responding, a SMTP success status is returned. Otherwise, you get some status such as 251 (User Not Local, Will Forward.)

Then, a threaded email client. It would help to have the convention that an email with subject but no text, text but no subject, or text the same as subject is displayed like an instant message.

All of these can be done independently, and are backwards-compatible. If you have all of them, email is equivalent to instant messaging - you can tell if it was delivered, it goes through immediately, and it works like a conversation.

Of course, there's no big-money startup in this.

> Then, a threaded email client.

Just to be sure: you are aware that that is a standard feature of email? There are headers that provide threading information, at least if you use non-crappy user agents.

> It would help to have the convention that an email with subject but no text, text but no subject, or text the same as subject is displayed like an instant message.

No, that would be a terrible idea, you never ever should redefine the semantics of existing messages. If you want to encode a new type of message, use metadata to mark it as such, that is what those extensibility mechanisms are for. Add a header to indicate that the sender is following some new convention, or add a multipart/alternative MIME part with a new MIME type that encodes such messages, with a fallback text/plain part for clients that don't understand the new convention. But whatever you do, never encode new semantics in a form that already has an established decoding/interpretation/meaning, and also always do it in a way that provides as much of the same functionality as possible with software that doesn't know about your new invention.

I tried some similar things when making Zombition, but I found that blurring the line between email and instant message mainly results in a more complicated UI and more details for end users to remember. For example, with instant messages you generally have some status information about whether or not the other user is online, whether or not they may have read whatever you just sent them, and whether or not they're typing a reply. If you want to add that into a email interface, how do you indicate to your end users that another user will never have associated status information because their account is on a server that is incompatible with the method you proposed?

Plus the behavioral differences between email and instant message didn't blend well: was I going to wait in a mixed email/instant message conversation for a response, or was I going to move on to the other conversations in my inbox and send a reply or archive each so that I could move on to doing other things? And if I wanted to temporarily leave an instant-message-like conversation so that I could deal with some more email-like conversations, would the person I was chatting with know whether or not I would return?

Eventually I stepped back to look at what additional value a user gained by blurring instant message and email in this way. Fundamentally they're both asynchronous text-based communication, and the things you can actually do with them aren't strikingly different. The biggest difference is in the expectations that a user has for each.

Anyway I still wanted to "innovate in the email space", so I decided to go a more extreme route, and I made my own pull-based protocol (superficial similarities to IM2000 - http://cr.yp.to/im2000.html) that retains backwards compatibility with email while adding abilities like stateful plugins (JavaScript apps/widgets) that can share data through the new protocol, editing messages after they're sent (SMTP recipients receive the new version after the update), and inline replies to parts of messages to enable nested conversations. If you're interested, I posted it earlier this afternoon here (https://news.ycombinator.com/item?id=10693535).

It's also very hard to push the benefits of the integration without having the product on both sides of the communication. A similar issue popped up in Windows phone 8. They combined sms and Facebook messenger -- but if you

> inline replies to parts of messages to enable nested conversations.

Just to be sure: You know that that is really easy with email as it existed until about 20 years ago, including BBS networks, usenet, etc., and that it is still trivial with good email clients? That was a solved problem that made email really efficient, until some clueless developers and users came along and disinvented (is that a word?) the solution in order to make email easy to use (or so they claimed).

Actually I didn't know; thank you for telling me! (That also makes me realize that I could likely do it in a much easier way than I currently am...) What are the email clients that allow that kind of behavior?

Well, essentially, all of the traditional text-only clients do (such as Mutt, which is what I use), I think Thunderbird does as well, beyond that, no clue, but there probably are others.

Traditionally, if you chose to reply to an email, the mail client would put the mail you were responding to into your editor, with > marks in front of every line. The only socially acceptable way[1] to write your answer was to insert your responses in between those quoted lines, without > marks, and to delete everything of the original mail that was irrelevant to establish the context for the reply.

That's how easy it is.

Usually, a good mail client would also color quoted lines so that it's easy to see and read only the response lines in a received email, so you'd only have to read the quoted text if you weren't sure about the context.

Clueless developers and users around the turn of the millenium then somehow didn't understand the purpose of providing you with a quoted version of the email you replied to, and started putting the reply above the quoted mail, thus ending up sending a copy of the mail they just received back to the person they received it from ... and nobody seemed to notice how idiotic an idea that actually is, and so it became sortof the new norm in large parts of the internet to attach large blobs of completely useless and often close to unreadable text to every mail, up to the point where some people even invented justifications for the behaviour that mostly tend to describe how this "feature" allows them to work around the lack of proper thread handling in their mail clients.

Obviously, easy quoting traditionally relies on plaintext only mails with fixed (short) line length, but format=flowed encoding has since been invented (the original RFC is from 1999) to accomodate screens and windows of variable width while maintaining backwards compatibility with old clients.

[1] https://www.netmeister.org/news/learn2quote2.html

Ah right. I was meaning something more like this so that you can skip the quoting business entirely and have the visual hierarchy match the conceptual hierarchy: https://s3-us-west-2.amazonaws.com/zombition-static/20151208...

> Clueless developers and users around the turn of the millenium then somehow didn't understand the purpose of providing you with a quoted version of the email you replied to, and started putting the reply above the quoted mail, thus ending up sending a copy of the mail they just received back to the person they received it from ... and nobody seemed to notice how idiotic an idea that actually is, and so it became sortof the new norm in large parts of the internet to attach large blobs of completely useless and often close to unreadable text to every mail, up to the point where some people even invented justifications for the behaviour that mostly tend to describe how this "feature" allows them to work around the lack of proper thread handling in their mail clients.

Yes, I think the rationale must run something like "What if user A deletes the entire conversation they were having with user B, and user B replies to the deleted conversation, but user A doesn't know what the reply is about because they deleted the entire conversation, have a terrible memory, and are too embarrassed to ask user B to explain what's going on? Let's just send the entire conversation as quoted text every time."

> Ah right. I was meaning something more like this so that you can skip the quoting business entirely and have the visual hierarchy match the conceptual hierarchy:

Well, if it's supposed to be compatible with all the HTML crap out there, and even most MUAs' broken encoding of plain text email, it might be hard, but in principle, I'd think this should actually be rather simple:

Simply specify an algorithm that defines how to segment a text/plain body into paragraphs and how to thus generate MIME object-scope IDs per paragraph (by simply numbering them) and use that same structure to provide UI elements per paragraph. Then, if the user writes a per-paragraph reply, generate a multipart/alternative body, with one text/plain part in traditional quoting style, optionally with format=flowed, and one alternative application/fancy-paragraph-foo part which contains in some sort of serialization format references to the message+paragraph IDs, that paragraph's text, and the corresponding reply text. This way, a receiving MUA that doesn't support application/fancy-paragraph-foo will display an ordinary quoted plaintext body, and also, if a communication partner uses different MUAs, you can still reply to an email sent using an MUA without application/fancy-paragraph-foo support and still have the feature work if the reply is then read in an MUA that does understand it. Also, including the text of the original paragraph makes sure you can still display a message sensibly when the displaying MUA doesn't have the mail it's in reply to - you'd just have to be careful to not misrepresent the authorship information as reliable.

> What if user A deletes the entire conversation they were having with user B, and user B replies to the deleted conversation [...]

* g * (is there any way to escape asterisks to their literal meaning?!)

Well, the rationale I've heard is that it's easier to make sure new participants in a conversation can read up on what has happened so far ... which obviously would be solved far better by attaching a thread of message/rfc822 attachments when adding a new participant that the recipient could navigate using their MUA's usual UI instead of having to read a close to unreadable unstructured mess of emails.

You can do the first part with Haraka. For what it's worth. I started work on an imap server but decided there was little point as nobody needs a highly custom imap server.

Reply on news.ycombinator.com