Today I solved a twelve-year-old Outlook mystery | Hacker News Reader

Today I solved a twelve-year-old Outlook mystery

emails.hteumeuleu.com

152 pointsbjoko68 comments

I cannot believe that Microsoft Outlook, in 2019, is still using the broken word-based HTML rendering which is an insult to the whole industry and a pain for anybody in email business.

Is there a strategy behind this? Is Outlook on maintenance mode with no developers? Or is it on their best interest to keep it broken?

Google wants some of that action: https://developers.google.com/gmail/ampemail/

Wow, this sounds like a horrible idea...

I see no scenario where this idea could succeed. Most businesses put their mails through sandboxes and whatnot. and a mail that looked like a chicken once, will look like a pile of bones at the time it reaches the receiver.

Well, we had mails that contained VBScript. Outlook users can actually execute these. I think Microsoft changed policies to default disallow for a lot of VB components about a year ago.

Since office 2007 they use word to render HTML mail. Probably mostly because of security reasons because at the time their other option was IE which lead to many previous security flaws.

Word isn't much safer but at least it is under the Office team's control plus it's less widely deployed than IE was at the time. It also has way fewer features with regards to HTML rendering which helps reducing the attack surface.

On the other hand, as it's neither browser nor HTML editor, its HTML rendering capabilities are never improved because it's no priority.

And thus HTML mail is stuck at tables, font tags and spacer gifs for the rest of eternity. Thank you Microsoft.

(OTOH, whatever helps discourage HTML mails and encourage plain text is a good thing in my book. And now, get off my lawn)

> Or is it on their best interest to keep it broken?

I get the impression that most mail is currently sent using Outlook and read using Gmail, so there is kind of a detente where everything is just frozen.

So Outlook users send more email than Gmail users?

How does this work? I mean it implies in some way a difference in the amount of reading Gmail users do and a difference in the amount of sending Outlook users do. That sounds weird.

Businesses use Outlook and send to lots of customers who have Gmail.

Do businesses really use outlook though, to send to their customers? The place I'm at right now automates sending out lots of customer emails, but they don't use outlook to do it they use some automated tool, however the HTML provided is outlook compliant HTML because you have to have something that can also be read in Outlook.

> Do businesses really use outlook though, to send to their customers?

Yes. This might be hard to grasp from an SV web dev perspective, but not all industries are just about shoving ads at people. The company I work for does not do mass mailing. We use email to actually communicate with actual people who wish to exchange actual money for our product. This often involves months of back and forth discussion, because our product is highly customized.

But I'd guess that since individual emails to customers are more common with B2B than B2C (due to higher transaction volumes), those emails are often Outlook to Outlook, not Outlook to Gmail.

the company I'm referring to is one of the biggest banks in northern Europe and they do automated mails to people who want those mails because they've paid a lot of money for the service.

The company I worked at before that also did automated mailing was only doing it because lots lawyers and numerous members of some other professions that required alerts regarding the laws affecting their jobs had paid lots of money to receive that email (well lots of money for a service that included alerting by email among many other things), and that email was setup to handle Outlook requirements because most people who received it were doing so on their outlook mails, ditto the current banking solution - for us webmail must remain a secondary consideration.

Anyway, given your condescending response I must say you made a nice choice for a user name.

Outlook as referred to here is the rich client program, not outlook.com.

You can send/receive gmail using Outlook - in fact it's much nicer to read compared to Google's confusing Web UI, with the ... and things along those lines.

Outlook is incredibly painful to accommodate when creating responsive emails.

It requires workarounds which include nesting divs inside of tables inside of conditional comments.

I’ve done many things but coding responsive emails has pushed my cognitive resources to the limit. Because of outlook.

We're building a software tool that lets you create HTML emails visually without worrying about client specific rendering bugs like this.

It's due to launch in the next month. If you're interested email paul.odeon@milliner.app and I'll add you to the list to be notified when we launch!

After working on crafting such emails for a while for my previous employer, I was seriously considering starting a SaaS to do exactly what you just described. (I didn't start.) Best of luck!

> Or is it on their best interest to keep it broken?

It seems similar to a strategy they use more often, famously known as embrace, extend, and extinguish [1]:

  1. Use some standard/specification

  2. Accumulate lots of users with questionable strategies

  3. Extend/break/deviate from the specification

  4. Devs update software to cope with the undocumented Microsoft(c) behavior

  5. There is no point in following the specification or standard anymore, since it won't work in practice anyway. Microsoft effectively dictates what devs do.

[1]: https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

It’s amazing to think how much collective time is wasted due to this strategy.

The biggest problem for modern email is outlook.

I use Outlook every single day, and it is my favorite email client (RIP, Google Inbox) because it's simple enough and still quite powerful.

What is "modern email"? If it's HTML based stuff, then I don't want it. I like my communication simple, because that makes it more reliable.

I dunno about Microsoft's intentions, but it's in our (worldwide internet users) best interest for Outlook to keep the 'broken' renderer.

HTML email has some use-cases as long as the HTML used is very limited, and that's exactly what Outlook provides. There's however no justification to turn email into a full on web page like Google wants. The longer Outlook's renderer is 'broken', the less likely we are to get abominations like Google's AMP email.

Ultimately, email really is a document and should remain that way. This is compatible with HTML, but completely incompatible with Javascript.

Oh come on, don't you wish you could send a game using WebGL via E-Mail? Think about the possibilities! (To some degree I'm unsure if I'm really joking - I can image practical use, but I love plain text mail, which I can easily search, filter and render as I need it)

Ah, wonderful. We'll have animated, autoplaying 419 scams. We'll need Turing-complete spam filters, which will be exploited themselves. Just what we needed.

Edit: Wait a minute. Wouldn't these WebGL games need microtransaction ability? That's a must for any game company. Obviously we must be allowed to use the payment API in emails! So now the scams would be able to bill the user directly.

As long as they can mine Bitcoin there's no need for micro transactions. Just make sure that there is no save button and warn the user that closing the mail will lose their game state.

A more serious use case might be the architect sending models of the new building or the doctor sending MRIs. While those cases have practical issues like the recepient not being able to respond to it easily (move that wall!) and privacy, they also have interesting benefits like no need to login to some cloud and retention period controlled by the recepient. Sending as attachment of course works, but needs container formats to collect assets together and sandboxing in browsers for local files. As well as this being a media break, where I have to switch to a different tool to look at it then where the context is.

Your suggestion is pure genius. We'll just call it a roguelike and that would explain the 'no save button'. I'll be sure to recommend you at the next 'dark pattern' meeting! /j

As for the latter, if we're going in that direction, we might as well go all the way and implement some COM-like thingy for the web, so users can attach multiple whatevers and work on them in the email. This seems more efficient than turning each email to a unique embedded app which can embed only one type + maybe attachments.

Despite all this being possible in Outlook-land, everyone I'm familiar with prefers opening the attachment and working in a separate window, but I guess some people would rather work inline?

Maybe they like it since there is no alternative!? If there were a working way things might be different.

I meant that in Windows there is an alternative to attachments to an extent. COM allows an application to embed in another application. i.e. I can paste some Excel into Outlook and it will keep the table formatting and I can manipulate it in Outlook as if it were Excel. It's not the best implementation though (pretty bad on the other side), and there probably is a much nicer way to do it.

You could be right, with a better implementation there would be more that choose to work that way. I still think the risk outweighs the possible limited benefit though.

There was a petition at one point to have the rendering engine changed and Microsoft made some kind of vague promises to improve it. https://freshinbox.com/blog/the-outlook-team-reaches-out/

I would assume that it's not a simple technical change on their end.

Given that Microsoft and Outlook started the disaster that is HTML e-mail, why would you think that they'd actually fix the mess that they started?

Your comment made me check whether it was MS that started HTML e-mails. Well, this doesn't seem to be the case, unless I'm missing something.

According to wiki, only Outlook 98 added support for HTML emails[0]. It's not difficult to find Outlook 97 users asking how to support HTML[1], so the wiki changelog doesn't seem to be wrong. Outlook 98 was released in June 1998. Outlook Express doesn't change this picture - version 4 didn't have HTML email support[2], and version 5 was only released in 1999. MS support date appears to be June 1998.

Meanwhile, Eudora 4.0 release announcement[3] at December 97 already mentions HTML support. So at the very least, Eudora had it first.

[0] https://en.wikipedia.org/wiki/Microsoft_Outlook#cite_ref-11

[1] https://www.excelforum.com/excel-general/396121-outlook-97-m...

[2] https://en.wikipedia.org/wiki/Internet_Explorer_4#Bundled_an...

[3] https://www.qualcomm.com/news/releases/1997/12/08/qualcomm-s...

Rich HTML Messages - Users have the ability to view and compose rich HTML messages, using graphics and stylized text. Emails containing HTML look as if they would on a web browser and include graphics such as embedded images, tables and java applets.

Outlook Express 4 was bundled with IE4, which was released in October 1997:

* https://en.wikipedia.org/wiki/Outlook_Express#History

* https://en.wikipedia.org/wiki/Internet_Explorer_4

I was also on Usenet at the time, and the garbage that OE produced was atrocious. And let's not get into top-posting.

(Now get off my lawn.)

I got another one for you!

It's 2019, and Office for Windows still uses Internet Explorer to render add-ins built with OfficeJS.

This is slated to be switched to Edge (regular edge, not the Chromium-based build) with Windows 1903 and a newer Office version, but damnit if my life isn't going to be hell debugging IE for a couple months.

When I clicked the link I thought it would be about something that confused me for a while: What do people mean when they write a capital J in their emails? I eventually figured out that it's because Outlook automatically converts a smiley emoticon into a WingDings (an old Microsoft icon font from before that term became popularized on the web) version, which happens to have J as the character for a smiling face. View that on a machine that doesn't have the WingDings font installed, and you just get a capital J. Raymond Chen blogged about it too: https://devblogs.microsoft.com/oldnewthing/20060523-10/?p=31...

Someone even believed that the capital J was a Microsoft thing. He used it himself in his answers to Microsoft. Confusion and then hilarity ensued.

(Source: link in parent)

Yup, Microsoft jumped at just the wrong moment on this stuff.

Today you just go "Eh, text is Unicode" and any weird corner cases go in the Private Use Area bucket never to be seen again.

And in the 1980s you'd go "Eh, text is ASCII" and just tell office workers that too bad they can't have an emoticon in their email, try writing :) instead.

But just at that moment where Windows exploded and everybody was getting a PC with a graphical user interface there was that awkward moment where it seemed reasonable to act as though text was arbitrary indexes into a font and doesn't actually _mean_ anything except that a lot of the fonts map to ASCII. So now you can write [HN removes actually Emoji] but it doesn't mean [HN removed the same emoji here] it just looks that way on PCs with the exact same font.

And that mistake infected Microsoft Office and all of Windows. I wrote a presentation on a work PC in about 1995 and then my boss printed it from her work Mac, and on every single sheet of the presentation the trademark symbol was replaced by some other symbol Macs had because of this useless ambiguity.

There are so many things Microsoft was ahead of the times. For example, they had what are now 'Electron apps' in the 1990's (hta files). And how I wheep when I look back at the power of COM, how I complained about it back when I didn't understand it, and what could have if we'd had proper object technology today.

Windows also had some chat application, hidden within the Windows folder, that allowed you to draw comic book style panels and converse using those over a network. I have never actually seen it used live, but it seemed like an interesting concept.

Under the hood Microsoft Comic Chat was a relatively generic IRC client. There's an interesting community of folks still randomly using it and you can find questionably legal, definitely unsupported downloads in places.

Honestly Comic Chat is one of the best things to ever come out of Microsoft. I miss it.

> power of COM [...], and what could have if we'd had proper object technology today.

Editors which can always crash, because they are as stable as the least stable plugin? Formulas you can no longer edit because you do not have that specific formula editor installed? Completely closed and proprietary binary storage formats? Security as weak as the weakest app installed on your machine?

Granted, in the 1990's, COM was probably a pretty good thing -- it was very fast and efficient, at the expense of stability. Today, I'd prefer different tradeoff -- stable, secure and open; and if I have to wait 5 seconds for my auto-rendered markdown to get updated, it's OK, I can do so.

Unicode has a couple of inter-related symbols blocks that preserve (almost) everything in Wingdings and Webdings, as it should. I'm surprised there aren't more "converter tools" between the original font encodings and Unicode preserving random usages of those *dings fonts. Exchange could have done that for particular Outlook clients. Word could prompt an option to optionally convert scattered usages in old DOCs.

(Office has at least moved to full Unicode rather than random Wingdings/Webdings embeddings in the last few versions, but given the long tail of the Office install base it will probably still be something like a half-decade to a decade before we see the last of the random dings.)

This confused me for a long time. I thought the person was signing off their email with their initial - which happened to begin with J.

I'm always annoyed by Outlook's HTML, I remember when it set Calibri as font and the fall back was "sans-serif" which would not really work.

It really made me wish that HTML in emails would vanish, just vanish and never come back again. Sadly this won't happening.

I'd be down with them either vanishing, or at the very least being compliant with a standard html5 spec. The in between state they're in now is the worst of both worlds.

Which HTML 5 spec is that exactly?

HTML 5 is a "living standard", which is a horrible idea for documents that might be archived for 10 years, like email.

Note, I'm not condoning the status quo, but let's think about what we actually want here.

A proper browser can display all HTML documents crated like, ever, if you can live with some imperfections that are there to deal with improper browsers in the first place.

I mean, I get your point, and it makes sense, and for a similar reason we didn't get WebSQL. But I think HTML is special in this regard. Everyone just accepted the complexity of dealing with all sorts of documents.

HTML 5 maintains backwards compatibility. HTML from 20 years ago still works today.

I would be happy if there was a subset of HTML for email. Look at the HTML Gmail produces, which is essentially just <div> or <p> in the default mode and uses <span> for changes of formating and <blockquote> for replies. This would work for propably 90 to 99% of all hand written emails in Outlook or any other email client.

> Outlook does support @font-face declarations (but doesn’t load distant fonts). Because Outlook can’t display that font, it falls back to its expected mso-generic-font-family value. And because it’s not defined, it falls back to its auto value, matching the roman font family, thus falling back to Times New Roman.

It also seems that mso-font-alt can be used in a similar way:

https://litmus.com/community/discussions/36-outlook-and-fall...

And there are many more mso-prefixed options available, though documentation is scarce: https://litmus.com/community/learning/8-outlook-overview

Generally it seems like this isn't very well known. Hopefully this gets a signal boost and best-practices articles get updated with the advice.

> I hope you enjoyed this read as much as I enjoyed doing this research.

Redmond's quirkery has been a pleasure these decades, for "yelling obscenities" values of "pleasure".

Maybe I’m dense and I couldn’t draw this conclusion — were those properties always available but we’re never documented? If that’s true it seems crazy to me that Microsoft couldn’t just say, “Hey I know everyone is mad about font-face not being respected. Here’s this property you can set and all is well.”

If you had passed a supported font to the font-family tag it would have rendered it (it will not download a webfont). Since people really don't like seeing square boxes instead of characters windows tries very hard to find a font that will draw the character. Based on historic precedent Times New Roman is normally chosen.

You'll note the fallback-families he lists are exactly those supported by the GDI LOGFONT structure which dates to windows 3.0. This is old creaky stuff we're dealing with.

https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns...

I concur it's crazy. Same thing happened with Internet Explorer. Back then, you could not have 5, 5.5, 6 together on one Win98. When installing later versions, they would replace earlier versions. But as a hobbyist I needed all to test my Web sites. The recommended work-around was VMWare. VMWare was not installable on Win98, so I tried Linux as host OS. That was the pivotal point that got me into my serious programming career.

Years later a hidden API was found out that made it possible to install the browsers in parallel. http://web.archive.org/web/2007/http://labs.insert-title.com... If Microsoft had documented that for the public, they would have saved Web developers thousands of man-years of work.

There is yet another gem hidden in that post:

    src: local('Pacifico Regular'), local('Pacifico-Regular'), url(https://www.caniemail.com/tests/assets/fonts/pacifico-regular.woff2) format('woff2');

Basically... isn't this a way to bypass the default external resource loading block for mail clients other than Outlook, meaning that this could be (ab)used for stealth "read receipts"?

Outlook can't even display raw TEXT consistently. We have a confirmation email that goes out when somehow submits to our contact us form.

In all email clients except outlook, the footer is on two lines (because there is a CR there).

line1<CR>

line2

In outlook (and only outlook)?

line1line2

The rest of the email is fine, just these two lines of the footer are smooshed together.

Have you tried using <CR><LF>? Windows notepad was never able to display linefeeds with only <LF> (or only <CR>?) and needed the <CR><LF>. Maybe this would solve the problem.

It's a feature. Outlook helpfully removes "extra" line breaks from plain text email, and there is no way for the sender to prevent this. One can only assume the developers never communicate source code snippets using their own software.

https://support.microsoft.com/en-us/help/287816/line-breaks-...

Dear Microsoft: line breaks in my preformatted text are not "extra". I intend every single one. You may believe you know better than me, but you do not. Stop trying to nanny, you are very bad at it.

Window's line separator is actually two charaters, CR LF ( 0x0d 0x0a).

This is a fundamental difference between Windows and other operating systems. There's unix tools available to convert between the two formats (dos2unix & unix2dos).

That’s an SMTP requirement too.

>The following line especially picked my interest

FYI, it's "piqued my interest."

Well, it’s not a bad thing that it doesn’t load remote resources...

I could be missing the plot here, but I always got the impression that Outlook limits (sets) the fonts and colours in order to make the emails look like they "were sent by MS/Outlook". But again, this may be my idiosyncratic MS experience.

I wonder, does Outlook 2019 have the "system" font glitch from Win 3.xx? I think I'll be sending out all my HTML emails in either System or Courier from now on; just to pull peoples' legs.

What’s the glitch? Microsoft has a font named System and if you ask for it you get it?

I guess they're talking about this.

https://medium.design/system-shock-6b1dc6d6596f#.v0lk6vj4x

Right, so not really a "glitch"

>Dormant, that is, unless someone asked for it with CSS. Which we, inadvertently, did.

It boggles my mind that not once did someone load up the "Medium 2.0" redesign on a Windows computer before launching it. Or even render it out from browsershots. Is cross platform browser behavior consistent enough that people don't do this anymore? And IE/Edge testing was just not on the list?

For reference if anyone wants to use the OS's font like this, "-apple-system" now has an unprefixed implementation as "system-ui" avoid the Windows font collision. Chrome/Safari support it, Edge will once they're running the Chrome engine, but Firefox doesn't yet.

"It boggles my mind that not once did someone load up the "Medium 2.0" redesign on a Windows computer"

It says in the article that it was only certain OS, browser combinations. Don't know how unusual the combination was though.

Ah you're right, I'd missed that. I wonder what versions.

> This morning I was doing some tests in Outlook on supported styles for HTML lists.

"Today I was doing unpaid QA on a widely reviled proprietary Microsoft office app ..."

Reply on news.ycombinator.com