Dear Email Industry, We've Got a GDPR Problem | Hacker News Reader

Dear Email Industry, We've Got a GDPR Problem

jacquescorbytuech.com

62 pointsiamacyborg89 comments

For those who, like me, needed to click on the article to understand what the "email industry" is: the author refers to the email marketing industry. This isn't a problem with email, but with email marketing.

Transactional e-mails also use tracking.

They don't need to, so if the GDPR actually becomes enforced it's trivial to disable tracking on those.

Indeed.

I use Sendgrid for transactional emails and have disabled all tracking.

Email is hardly a guaranteed delivery system by nature, I do not need to have proof of delivery or proof of opening by tracking systems or even the logs (beyond needing to debug if a customer asks why they didn't get an email).

In fact, all I really need is the ability to resend a transactional email on demand, or the equivalent action I can communicate (in my case I run forums, and a transactional email would say "someone replied to your comment" and I can point at a page on the website that has the feed of recent actions).

Tracking pixels in email should be worthless, because loading external images should be disabled by default in email clients.

Instead, all email clients I’ve ever seen default the other way. Maybe Apple would some day be willing to make its client more secure in the name of privacy by flipping the default. I’m sure they’d face a backlash, though.

I'm pretty sure Thunderbird defaults to not load external images but I only install it every 5 years so don't quote me on it :-)

I'm very sure thunderbird defaults to blocking external images (but there is button to download if you want), because I'm using it daily exactly for this reason. I also have my mail server from small local provider and it has several web mail clients available, each one of them is setup to not download external images.

> Instead, all email clients I’ve ever seen default the other way.

What E-Mail clients are those? Both I use (Outlook for work, Fastmail privately) do exactly that, and neither of those are small. The only one I know that’s different is GMail, but well, Google and Privacy.

As far as I know, GMail actually downloads the image as soon as the mail arrives on their servers, accomplishing the “making tracking useless” task in another way.

This is not correct. Gmail's cache certainly obfuscates some data, such as the IP, but email pixels will still tell you when an email was opened and by who.

I just recently saw a completely opposite suggestion that would also render tracking pixels (mostly) useless: the e-mail server could load, and cache, all the images when the e-mail is received, regardless of whether it's ever opened or not.

Ohh, that sounds perfect for DOS attacks. A clean reflection attack vector operating on giant assets.

Even better DDOS, just spam lists of image links to various email providers and watch your target collapse under all the requests.

What would be the difference from normal hotlinking in a standard email? The vector in both cases would be being able to send a million emails; a reflection attack with spam emails as a vector would be just the same without provider servers doing anything special. MUA opening images would do the attack.

Actually a provider could cache the hotlinked resource and remove almost completely the reflection.

Using only plain-text emails solves the problem nicely. Who'd even want to ever use HTML mail other than shady or clueless or annoying people? Just use plain-text only and the problem is solved.

Yeah, the typical HN/usenet/ML solution to everything: just use plain text, it's good for everyone and nobody needs anything else. Newspapers and books always looked like an RFC document anyway, right?

How? If you send an email with an image to 15k recipients in a mail server the image would be fetched once and the recipients would only get the cached version.

Image URLs often contain identifiers that are unique to each recipient.

So this would only dissuade that practice as it causes a DDOS against those who do because it causes additional requests and prevents caching.

The image url can point at an innocent third party.

Which innocent third party provides thousands of unique image URLs?

https://example.com/img.png?x=1 https://example.com/img.png?x=2 https://example.com/img.png?x=3 https://example.com/img.png?x=... https://example.com/img.png?x=1999 https://example.com/img.png?x=2000

You need but one image, to craft an arbitrary number of unique urls using a querystring.

In theory one could also use all permutations of uppercase/lowercase letters in the path to the image. Most webservers are case insensitive so all will yield the same result but I believe the http standard considers the path to be case sensitive so all those urls would be unique.

As I said in another comment, crop all the parameters from the URL, and the problem solves itself. Path is case sensitive, the domain must be lowercased anyways, so no problem either.

Crop the parameters and you have no reason to assume it points at the same image. Good luck explaining to your customers what happens when someone finds a way to effectively poison your non-unique image cache with something offensive.

> Crop the parameters and you have no reason to assume it points at the same image.

Well, that is going to be a problem for the mailer. I'm totally fine with banning dynamic parameter dependent images.

> Good luck explaining to your customers what happens when someone finds a way to effectively poison your non-unique image cache with something offensive.

I would say it is the email marketer fault for using unsupported parameterized images. I cannot image a legit use for that, and many evil spammy ones.

> I would say it is the email marketer fault for using unsupported parameterized images.

The problem is this would not just happen to e-mail from email marketers, but also between regular users, and it would take just one particularly nasty exploit of cache poisoning of urls to some site with user-generated content before you suddenly have the press asking you why some innocent picture sent by someone underage to someone else underage was replaced by your site by hardcore porn - or worse.

I've run a webmail provider. I've seen the amount of abusive bullshit spammers and scammers do whether for profit or for fun or to get back at someone. It used to be my job to find these kind of issues before bad guys did, and one thing we learned very quickly was that every little thing like this would instantly have people probing it for ways to abuse it to cause grief for someone else. Or for us.

If you were going to ban images parameterized by URL parameters (and that would not ban parameterized images, just reduce the number of sites that could be attacked), the only viable choice is not load them at all. Just stripping the parameters would be an absolute disaster and wildly irresponsible.

So your solution is basically to throw various standards overboard so that spammers cannot generate more urls than there are images on a specific domain. Isn't this cure worse than the (at this point mostly hypothetical) disease?

Crop all the parameters from the URL. Problem solved.

> Just stripping the parameters would be an absolute disaster and wildly irresponsible.

It's not worse than allowing a randomly selected subset of HTML in the emails, and nobody is saying it is an "absolute disaster" or saying that google or Microsoft are "wildly irresponsible". The mailers and people will get used to it. As they always do.

The email server does not need to load all the images at once. If it receives a billion emails, it can retrieve linked images a few per second per domain. Or on demand if the email is opened. The email server does not even need to completely drain the queue and load all the images, just images from enough emails to bollocks up the results is enough. The end result is the same, in that the sender does not know if the email was read by a human or not.

> just images from enough emails to bollocks up the results is enough.

A large enough proportion of opens either does not trigger an image retrieval or triggers one but the user won't read the message anyway, so open rates have always been approximate. The time of day, the news, the weather all distorts the data so much anyway that the only sensible use of the data is trends, so it'd take some effort to distort the data enough for it to be useless.

Your best suggestion I think would be to rate limit it, with the caveat that if you want to immediately deliver messages you'd need to also open on demand, and e-mail opens are extremely spiky so you might struggle to smooth out the opens entirely, but it might be enough to make it useless enough for people to ignore the data.

This doesn't actually buy anything since with external images blocked the sender can just attach all required images to achieve the same effect without the receiving server having to parse HTML and speak an additional protocol.

gmail does this I think

Gmail doesn’t do this.

https://blog.filippo.io/how-the-new-gmail-image-proxy-works-...

It's bad for spam because spammer can recognize what address is alive. Possible solution is open all images including mails sent to non-exist users (wildcard)? Looks like much waste of resources.

Agreed. It's one thing that made a great first impression on me with protonmail, as external content is disabled by default.

K-9 Mail defaults to asking the user if he or she wants to load images, at least on my Android phone.

Remote images are disabled by default in thunderbird, the Gmail webapp, the Gmail android app, the fastmail android app and the fastmail web app. Gmail makes an exemption for people manually added to your contacts, and they all have an option to load images by clicking a button which then gives an option to whitelist the sender for images.

The Gmail iOS app is the only example of an email client I have handy that doesn't do this by default and since that's signed into my work gapps I can't rule out that being something my employer has changed in the config.

> the Gmail webapp, the Gmail android app

Sort of.

I believe images are off by default if you add a non gmail address to gmail via imap/pop but @gmail addresses in gmail will default to images on.

Is this a g suite/consumer gmail difference, or some sort of grandfathered default? Because that's not what I observe on any of my gmail accounts.

Hmmm, not sure. I've not created a new consumer gmail address in over a decade. New gsuite accounts definitely have images on by default though.

I use Gsuite daily through work and images are off there by default. Maybe it's a company default?

As far as I remember when configuring my GSuite account I as the sole admin was able to configure images to be default of in mails.

But maybe this is a regional thing as well.

Also Apple mail will load the pixels when forwarding an email, no matter what the setting is.

Dear Email Industry, I hope you become obsolete real fast.

Does anyone really read those e-mails websites send them? I can't be the only one looking for the unsub link immediately because you accidentally left the check 'subscribe to our newsletter' on when ordering something.

You know, those checks that are turned back on when you post the form but are redirected to the form because of validation errors.

> Does anyone really read those e-mails websites send them? I can't be the only one looking for the unsub link immediately because you accidentally left the check 'subscribe to our newsletter' on when ordering something.

Yes, a few people read and engage with them. They're so inexpensive to send that many companies can justify the cost with a conversion rate of < 1%.

I actually don't mind them. Most of the emails I get are from retailers where I am likely to buy something in the future, and seem mostly to be in the form of a deals-leaflet. A nice quaint bit of somewhat useful, traditional advertising.

I can't think of a single "newsletter" that I've received in the past two years that I actually read. Usually my response is "huh, did I forget to uncheck a box? Unsubscribe".

> Does anyone really read those e-mails websites send them?

Hundreds of millions of people. Every day.

The problem is more basic. EU companies illegally transferring billions of email adresses to Mailchimp.

[Edit] The ECJ voided Privacy Shield because it did not protect the data of EU citizens.

The way forward would be Standard Contractual Clauses (SCC)as a tool the ECJ said.

The EU comission formulated (an instance of) Standard Contractual Clauses. These have not been accepted by EU data protection agencies (yet). Until they are accepted they do not protect against being fined.

Those are for every third country.

In the special case of the US currently you can't create SCCs that work because the EU citizen has no lever against three letter US agencies. Until the US changes it's position here, you're not safe with SCCs. Private contracts can or can't safe you. e.g. if I have a contract with someone to steal something togther, the contract doesn't make it legal. Although it is legal to have a contract.

Illegal in what sense?

Not the parent, but here's my take:

Email addresses are PII and Mailchimp is a US company. With the death of privacy shield, handing over your customers' PII to an American company is a gross violation of the GDPR and its implementations.

An EU company sending email to customers using American services would be a legal minefield if the GDPR would actually get enforced. As far as I know, no data protection agency has looked into the practice as of yet. Of course, that could all change very quickly.

Personally, I think the use of American services such as AWS, Azure and GCloud should be looked into first, though. Sharing a list of email addresses is nothing compared to placing your entire customer database into the hands of Amazon.

It really isn't a mine field.

You can simply sign a Data Processing Agreement (DPA with MailChimp containing Standard Contractual Clauses (SCC) and you're good to go.

No you're not.

Standard clauses CAN be a solution if they address the problems with transfering data. Currently you can't have Standard clauses with the US, because EU citizens have no say against the NSA.

You absolutely can and it's the official solution proposed by the Court of Justice of the European Union since July 16th 2020, date of the Max Schrems II decision.

In theory.

The court clearly stated that the data exporter has to suspend the data transfer if the recipient is unable to comply with that contract (which must ensure the level of protection required by EU law). This is currently not possible for an US entity in practice, as the court also found, because US law does not grant non-US-citizens actionable rights against US authorities in that matter.

Official summary by the court https://curia.europa.eu/jcms/upload/docs/application/pdf/202...

Judgement itself http://curia.europa.eu/juris/document/document.jsf?text=&doc...

Deleted my comment because yours is much better.

And in practice the website of the Court of Justice of the European Union loads YouTube videos so I'll follow what they are doing as an example and justification.

The whole thing is a minefield - I think the idea is great - but the whole implementation is just such a fudge, and it came in with no real clear guidance on how things should work. I think that's why we don't see a lot of enforcement, unless it is a blindingly clear violation.

German data protection agencies announced last week that they have compiled lists of companies because they want to start enforcing this 2021.

"German companies are threatened with stricter controls because of the transfer of personal user data to the USA. The majority of the German data protection authorities are participating in a task force headed by Hamburg and Berlin, "which coordinates the implementation of the requirements of the Schrems II ruling," said the Hamburg data protection officer Johannes Caspar on request from Golem.de. The authorities wanted to randomly select and write to companies nationwide "for which there is reason to assume that they use service providers from third countries". [German] https://www.golem.de/news/datenschutz-task-force-will-nutzun...

> I think the use of American services such as AWS, Azure and GCloud should be looked into first, though.

Many EU companies use the EU hosted versions of these services. For example, I know of a number of EU companies that host their AWS stuff in eu-west in Dublin.

1. Yes.

2. https://en.wikipedia.org/wiki/CLOUD_Act is not tested yet with the ECJ.

As long as it is being disclosed, then there is not a massive problem. They have a Data Processing Addendum [1] that covers GDPR stuff, and the whole Privacy Shield stuff from last year doesn't matter too much because Standard Contractual Clauses are still valid (which that DPA is a part of [2]) so nobody is doing anything illegal if they are following the rules that were set out.

Obviously there are still plenty of companies that /don't/ follow it, but that doesn't rely on Mailchimp being involved.

1. https://mailchimp.com/legal/data-processing-addendum/ 2. https://mailchimp.com/help/mailchimp-european-data-transfers...

Edit: That's my understanding from reading their docs last year, anyway

Privacy Shield is void.

Standard Contractual Clauses need to be validated by the EU on individual bases, those companies have with companies in the EU are most probably not enough - but this is not tested yet.

At least for Germany it's clear that Standard Contractual Clauses in the way they are now, are not enough. Because they don't solve the problem of the NSA grabbing data without EU citiziens having any rights.

German data protection agencies have started a project for 2021 where they have compiled lists. I would assume everyone using MailChimp will get a mail from an agency this year.

Ok, I am glad the author has figured it out (2 years too late) but I have another question:

Why are they even using tracking pixels, I cant remember when I had some email client that was showing the 3rd party images in emails?

I thought this is a dead "technology" for at least a decade?

What do you think is dead? Email marketing or email tracking?

In either case, both are very much alive and doing better than ever, though in the latter case this is not a good thing.

No, I meant specifically tracking using url to images on 3rd party servers. I have just checked outlook and evolution and both are blocking it by default, same with thunderbird and nine mail O.o

Sadly, nope, still very much a thing that is happening.

I sometimes cannot tell if HN users are being sarcastic, or are genuinely out of touch with how people use technology.

You're not a typical user if you have Thunderbird, let alone Evolution. Gmail is the de-facto standard provider for personal email. People use web clients on their computers, and either the Gmail app or Apple's apps on mobile. Those load images by default, and render HTML. If you're using Thunderbird on your PC or K-9 on your Android, you're in a minority, and your habits don't reflect the vast masses targeted by marketers.

Gmail for web and android does not load remote assets by default. images have to be either attached or in data URLs to load from most senders (only your contacts and whitelisted senders get a pass).

So what you want to say is that glitch that was fixed 20 years back in all major email clients is back and the "de-facto standard provider for personal email" has neglected it?! Probably deliberately.

Nice. With a huge facepalm. I wonder if they will also add support to run .pif files. /s

I really didn't know that, haven't used gmail from the times when it was invite only, I prefer better email clients than the default one (now that you have told me this - even more) and I am happy camper with my own mail server for even longer while on the other side I don't have time to track how far the Idiocracy[1] has progressed

It was 9 mail[2], not K-9. Firewalled to be able to access to only my and company domain. But yes. I am not typical user.

[1] https://www.youtube.com/watch?v=sP2tUW0HDHA

[2] https://www.9folders.com/en/index.html

Gmail at least still shows them by default, and that gets you most personal email users in one go. I know Apple Mail does as well.

I’ve created https://ohmysmtp.com - a transactional email provider. One of the things we’ve specifically done is not include an option to track opens. I think we’re the only provider out there that doesn’t do this, and most if not all default to tracking opens.

Of course it would be useful to know if a user has opened a transactional email in some cases, but it’s really not necessary.

My personal gripe with tracking pixels is the UX, users have no idea that they are being tracked inside their email client. If there was a big button that said “tell the sender you’ve read this” would any user actually click it? Almost certainly not

Previous discussion: https://news.ycombinator.com/item?id=21024926

And indeed, (2019).

Also a related discussion from today, Spy pixels in emails 'have become endemic': https://news.ycombinator.com/item?id=26162513

>...tracking pixels and tracking links...

I consider the fact that email clients can ever be in a mode where such things can work quietly in the background the actual problem here. A program should never leak information in a way that is out of the control of that user. Perhaps we could try applying the GDPR to the creators of such clients.

God bless GDPR. I only wish it was enforced more. Small businesses seem to not be aware or are faking it that they have legitimate interest for the most basic information. Gyms and similar facilities have been very adamant about me signing the consent form. They tell me "and now just the GDPR form" and I tell them "oh, it's ok, I don't want to sign it" and they keep insisting "I have to". It's ridiculous

The regulators here in Norway has gained momentum and have this year already fined as many as they did in all of 2020.

You might like https://www.enforcementtracker.com/ , it's pretty cool to see all the enforcements so far.

You mean the lack of enforcement? I don't see neither Facebook nor Google being fined for their non-compliant consent prompts nor analytics.

Google got fined (peanuts for a company of their size) for other stuff, and Facebook only got fined the equivalent of a few cents once on a technicality.

From that list, Google got fined 50 million euro in France for "Insufficient legal basis for data processing". That's not "other stuff", but is peanuts for a company its size.

https://www.cnil.fr/en/cnils-restricted-committee-imposes-fi...

You should report these facilities to your local DPA so something can be done against them.

I sort of feel this is looking at some shadows on the window curtains of the house and thinking there is a hell of a party going on inside. Which given the way things generally work is probably correct but not necessarily so.

Example - if you send a tracking pixel with every email you send out and you track those pixels to see how many of your emails sent out were read without connecting each tracking pixel with the user account I don't think you're in violation of the GDPR.

If you send out a tracking pixel that is

1. designed to track user reading of emails per user

2. and the user has opted out of the tracking,

3. and the backend implementation is look up tracking pixel id and lookup user assigned to that id and look up if user consented

4. if not consent do not track user read email.

I'm not sure that scenario is against the GDPR either, although it may be that the company is told at some point - do not send out the pixel if the user has said they don't want to be tracked, in other words you will not be allowed to stop tracking at the backend, you must not send the image with any non-trackable email that would you allow you to potentially track that email even if you never follow up on that potential (which would really seem to be a good policy to have)

> 2. and the user has opted out of the tracking,

> do not send out the pixel if the user has said they don't want to be tracked

Per the GDPR, they need to opt in, not opt out.

This.

But also, most email platforms don't have the ability to add a pixel on a per-recipient basis. This is a clear failing on the part of these tech companies.

>most email platforms don't have the ability to add a pixel on a per-recipient basis.

right, which is why I assume that they would add the pixel and the company would decide to not track the non-consenting users at the backend. Everyone has the pixel, but only consenting users get tracked by the pixel being requested.

That sounds more open to failure than simply not tracking people who haven't opted in though.

yeah but there is such a thing in computing as legacy decisions and I'm pretty sure the addition of tracking pixels is legacy in that they were added before GDPR back in the dark ages when everyone thought surely we'll be allowed to track every person and sell their data as much as we want forever!

So now you find out you're not allowed to track if someone hasn't opted in.

But because of reasons it is difficult for you to fix the thing that puts in the tracking pixel of pixels. So you leave those in place, and you do a check on the server when that pixel gets requested as to whether or not that user should be tracked, and figure that will be good enough.

yes, but sometimes when one is writing fast one makes slight errors that don't make much difference overall, thus:

If you send out a tracking pixel that is

1. designed to track user reading of emails per user

2. and the user has not opted in to the tracking

3. and the backend implementation is look up tracking pixel id and lookup user assigned to that id and look up if user consented

4. if not consent do not track user read email.

I'm not sure that scenario is against the GDPR either, although it may be that the company is told at some point - do not send out the pixel if the user has not said they agree to be tracked...

so, I don't think that it made much of a difference in my argument.

Reply on news.ycombinator.com