Hacker News Reader

It doesn't seem that complicated to remove a single entry from the e-mail distribution list of millions that's going out in one to two days. Harder problems have been solved. This doesn't adequately explain it.

The reality: it's financially a waste of time for a company to create an A+ off-boarding experience. Getting 1-2 extra emails isn't the end of the world: it's good enough for the company, you get off the list, they don't have to think about that unprofitable problem any further.

Most companies don't put much effort into off-boarding. Some go out of their way to make it even less palatable. But for removing yourself for a mailing list, I don't consider this to be the worse. Things surrounding payments are much more important: if I cancel service, you better not continue charging me.

Most people don't consider the full lifecycle of anything, and I wish they did. We've created a society, for example, that consumes disposable plastics, as an ordinary daily activity, and that ends up in our waterways as microplastics. It's easy to make things. It's an order of magnitude more difficult to manage the full lifecycle of the thing.

The technology behind it all could actually function perfectly but in reality an overworked marketing intern grabs a csv file from their desktop, opens it in Excel, compares and manually removes the unsubscribes (from another csv file that's delivered to them daily by email because that's easier) because "that's how it's always been done around here", then saves it as "June email push (1)(1) copy 2 (1)(1)(1)(1).xls" on their desktop, deletes all their subscriber lists in Mailchimp, as per the usual procedure, and imports "June email push (1)(1) copy 2 (1)(1)(1).xls" from their desktop, and sends out whatever.

The fact that the company is run so inefficiently seems to be a separate problem from the technical feasibility.

So much this. These kinds of workflows are far more common than you would think they’d be, looking in from the outside.

This is true even if your company has in-house developers. At the small company I work for, we have barely enough developer bandwidth to cover most things related to our core products as it is. Improving internal support processes just isn’t even on the radar for us as a dev team. So those things either get contracted out (where we as a dev team may have limited or no evaluation input with regards to the quality of the solution), purchased (again with little or no input from us as devs) or are built ad-hoc by the less technical side of the business.

Companies have a lot of internal moving parts and resource limitations that together lead to these kinds of things.

Ah, i wrote my message out to my Kafka queue? Oh, there's not an easy way to remove it? Great, let's rearchitect our system around a different system that makes it easy to remove items from a queue once we've sent it out. Oh, there is none? Great, let's hire a bunch of experts who can write such a system from scratch that competes with the mainstream queueing systems out there. Or we could just say queued messages are queued, and tell our customers they'll stop receiving emails roughly a day from now.

If you (generally, not you specifically) architected a system that's brittle in the face of new requirements, that speaks to the resilience of the architecture to withstand change.

> Getting 1-2 extra emails isn't the end of the world: it's good enough for the company, you get off the list, they don't have to think about that unprofitable problem any further.

If I receive any further e-mails after unsubscribing, I mark them as spam. I don't think I'm the only one who does. That should be some incentive against sloppy unsubscribing mechanisms.

You've unsubscribed. You're no longer a customer. Why should they care? (Other than if they want to be decent human beings.)

Because 1 in 3 email adresses is @gmail.com and google learns very fast from the Spam button. So if you want your emails to arrive to the subscribed people...

Not really my place to judge how your system is built, but double-checking an email is still subscribed just before you push it out from the queue isn’t impossible. At your volume, this will end up around 20 primary key requests per second, which is a decent database load, but then again for a company that needs to contact 2m customers per day it sounds like this wouldn’t break the bank.

Not letting people unsubscribe quickly is a conscious, commercial choice/tradeoff, not some kind of hard technical limit. Legislation can push this easily in a different direction.

Oh, of course, double check every single email just before it is sent, that will not impact the server very much. Except it will. By a lot.

Thank you. You said what I was trying to, concisely.

Nah it doesn't. Cache the list of emails that have unsubscribed, your dispatching process will filter out from that list.

Let's say 10million have unsubscribe, at 128bytes per email. That's roughly under 1.5 gigs of ram to store all those in memory.

Sending 2million emails daily comes out to an average of 23 a second. You can search 23 times through that list pretty quickly with one extra CPU core.

Pretty sure you could check a email in a database more than 23 times per second these days.

If it was sane company, you could. When I was doing some support for less sane companies, sometimes their "servers" were less powerful than my phone (hey, it still works after 10yrs, why change it). When my current company made product, everyone in such less sane companies was amazed that we could produce monthly route reports for fleet cars in seconds instead of minutes (and as devs we thought it was still not optimized enough).

You don't need to double check, you just have a way to remove emails from the queue. And doing simple removal from the queue only when someone unsubs is much less work than checking every e-mail in the queue against some central subscriber list, before sending.

Presumably people are unsubbing all day.

If n is the number of emails and k is the number of unsubs, it adds up to O(n) to check the table of unsubs before sending vs. O(k*n) to scan the queue after every unsub.

Or, to be less computer-sciency, a task that scans the queue of emails every time someone unsubs would be a pain to keep running. Elsewhere in this discussion the queue was described as a file. Who wants to scan the same file over and over all day?

Yes, but all that is easy to optimize. You can scan the mail file on entry to the queue and store file path/recipients to an indexed database. Then removal is just one SELECT query and unlink() call.

Well, actually no.

The vast majority of bulk email is done via 3rd party providers like constant contact or mail chimp because delivery is important.

You can't just check an address--you have to check against the particular mailing list, email address and account which means you'd have to basically have to have the database. Otherwise, if I unsubscribe from "Megous Monthly Mailer" I'll stop getting the announcements from my kid's school with useful info like what the first day of school for my kid is.

So you need to know the account, the mailing list that was unsubscribed, the email address. Of course, the system is architected for high-volume, and scheduled email. Delivery volumes can be in the billions per minute. Anything too crazy and your service level is destroyed.

Queues are designed for high-volume queuing, not search, or retrieval. There's no "query API" for rabbitMQ, for example.

All of this is easy if we hand-wave away the requirements for high volume email delivery.

I mean I agree but it's also possible to store a list of recently unsubscribed email addresses on the servers sending the mail out. Is it worth the trouble? Probably not.

At the point it's in the queue, it's already a composed email. Doing a file search and deleting the email file might be possible. I don't know how realistic that is though.

We built your customized email 4 hours ago. It's sitting in a directory somewhere waiting to be read and sent out to gmail. We're waiting because we already sent "X" emails to gmail addresses 3 minutes ago, and we need to wait "Z" more minutes. It might be in the list of emails to be sent as soon as the gates open up again. It might not be ready to go until another 50,000 emails before it have been sent.

So, let's say you've decided you don't want to get our emails anymore. Great. We've updated the appropriate tables. Next cron job will skip you. Now let's implement your criteria.

We've got to tell the MTA to figure out which is your email, and delete it from the queue. Hopefully we got to the MTA in time. We might not have. Remember the example I initially gave? You might be receiving the mailing 2 seconds after you unsubscribed. So we've spent some measurable time trying to prevent something that's already happened.

We don't have enough unsubscribers in general that this is going to cause a significant overhead. The thing is, I'm not even sure I can do that. Our routing application is a 3rd party commercial application that we don't have the source code to.

Not to mention, we're going to have to hop back and forth between machines as our databases are elsewhere. Not even sure they're in the same data center, now that I think about it. I really don't want to look up the NAT information to work this out for a simple internet discussion.

I'm gonna punt and say, "I dunno. I know how my part of the company works, and I'm still learning it. Maybe there's a better way to do it, and if you have information on commercial products that will do what you're stating is the desired goal, I'll do some research on them, and make recommendations to our company if they look viable."

I'm a C# developer. I'm still not quite sure how I ended up handling email. :D

If you were talking about the difference of a few minutes, then sure I'd let it pass. But hours? No, that's ridiculous. How long does it take to check if the destination email address of an outbound email has been unsubscribed? If it takes long enough to make it so you can't maximize your rate limit then you're doing it wrong.

> At the point it's in the queue, it's already a composed email.

So just...don't send it? I don't see why having done prior work to compose it means that you have to send it. You could argue that it's a waste of resources to have composed it and not sent it, but that's a sunk cost you can't get back, so why not just throw it away if you already know that's what I prefer?

> So just...don't send it? I don't see why having done prior work to compose it means that you have to send it.

So just heavily modify sendmail/qmail/whatever to maintain a list of email addresses for which it shouldn't send it?

Not associated with the grandparent, but the software likely composes and immediately "sends" the email, at which point it's going to be in MTA queue that's completely oblivious to the fact some email addresses have been unsubscribed since they entered the queue.

Overall, what you're suggesting sounds somewhat unreasonable. Once the email is "sent" (actually, added to MTA queue!), he might have no control whatsoever what the MTA does.

If you throw a stone, stopping it mid-air can be... hard. Perhaps it's doable. But is it a reasonable expectation?

I can see a case where the actual mailing is being handled by a delivery service and the ability to remove a single entry from a queued job might not be technically feasible because either the relay service is not exposing it to you or (and hopefully the actual reason) the delivery service only has the job that data stored in encrypted form and addresses are decrypted at time to send using a kms/kmip type of process.

I really don't think the possibility of receiving one more email rises to the level of needing new legislation and adding one more liability to help push people to moving all of the data and logic externally.

I don’t think it’s particularly onerous for a relay service to have an API for ‘remove all outgoing emails for this address’. It can even be hashed, because it doesn’t really matter what the address is, just that you know what mails belong to it.

I’m fairly certain that’s something postfix will do for you too.

Just because someone has unsubscribed from your mailing lists doesn't necessarily mean you want to stop sending them all mail. For example, we've got people who unsubscribed from our mailing lists but still submit support requests by email, using the same email that had been on our mailing lists, and they expect to continue to get answers to those requests.

This seems like a solvable problem? I would think that it wouldn't be hard to mark support emails in a way that didn't overlap with marketing emails.

In other words, We just don't care about users that unsubscribe (and why would you, really, you only care about profit) so we (unintentionally, maybe) engineered our systems so that unsubscribing is one of absolute lowest priorities.

Why add a ton of additional work and overhead to serve users that aren't likely to be customers? Makes no sense.

Ethics?

Seems pretty ironic how much effort was put into sending emails to these users in the first place if they "aren't likely to be customers"

When you want to cancel and Amazon order, do you expect them to chase down the mail truck to the retrieve the package?

Not solving nonproblems is how reasonable people operate.

In this scenario, I never willingly placed the order in the first place.

If Amazon started sending me a ton of useless random packages every week, you bet that I'd expect them to have a team of ninjas chase the driver down and forcibly remove my package.

Spam and marketing mail should be - needs to be - opt-in and illegal.

I would love to have an ACL on my physical mailbox, where all senders I have not explicitly authorized get mail returned to sender at their own cost.

If it takes up to 2 days for the system to send the first email then I think it's fair it takes up to 2 days for the system to send the last email. To some reasonable limit of course (i.e. just setting 1 year queue latency isn't a fair play).

Interesting problem. Of course I would still press the spam button upon receiving the second email and given that I use gmail, it would probably hurt your send rates.

Might want to fix that with a filter on the queues.

> However, we already placed you in the queue...

Well, at some point you took an architectural decision with an implication that if a customer requests to unsubsribe at time t, the request will be effective at t+N. This is a choice you made so the short of it is, it takes N becase it suits you.

If I haven't explicitly told you I want your email, and you haven't confirmed that choice,and you send me email then you are spamming.

In the UK if I haven't bought anything from you and I haven't asked you to email me and you email me then you may be breaking the law.

Out of curiosity which service do you use to send 2 million mails a day? If it's from your data center, don't those IPs get blacklisted too quickly?

There are whole businesses that exist to do this. eg Marketo, Eloqua, Responsys, Cheetah, etc.

And they send (and inbox) millions of emails by having reasonable open rates, not spamming, and respecting opt-outs.

Ahh “open rates”. The great lie of the email marketing industry. Those numbers that Marketo and company show are either completely made up or at least have hidden “scaling algorithms” applied.

All modern email clients block all image and CSS requests by default, and email security filters examine any linked URLs automatically. So there is no reliable “opened but did not click” feedback mechanism at all in modern email.

Set up a seed list of 100 email addresses you control, send through email marketing tool of choice. Open exactly 1 of those emails from a default client. See what lies their “open rate” dashboard gives you.

As was clear in context: Google evaluates the real thing for inboxing. It is relevant to which tab your emails get placed under. Hence all the advice to remove people who don't engage.

Whatever open rate stats are shown to marketers are irrelevant to google's deliverability choices.

Most of that question involves a level of detail I can't get into. But, we use the same batch of IP addresses for all our mailings. We have authentication set up with SPF, DKIM, and DMARC, and we're very careful on our rate of delivery, and as above, we respond quickly to unsubscribes. As well as the occasional report of our mailings getting labeled as spam.

We also partner with a reputation monitor that makes sure we're behaving, and they also let us know if we're approaching thresholds of not email friendly behavior. The only time I've seen that happen was when we did some IP consolidation and certain mail handling companies automated systems found our behavior anomalous, and thus was flagged suspicious. Since we knew, and our monitor knew this would happen, it was handled, and everyone (us, monitor, mail domains, customers) was running back to normal behavior within about two weeks.

In the end, no blacklisting from this, and none while I've been here. We've had a fairly high inbox delivery rate (based on our rep monitor, about 3% higher than the average) over the years, and about the only thorn in my side is Comcast, and even our reputation monitor has "special" metrics to measure their customers in regards to Comcast. :)

Why not use something like Amazon SES?

They know how to send emails and, most importantly, how to collect complaints and bounces feedback from email providers.

Ahh “open rates”. The great lie of the email marketing industry.

Those numbers that Marketo and company show are either completely made up or at least have hidden “scaling algorithms” applied.

All modern email clients block all image and CSS requests by default, and email security filters examine any linked URLs automatically. So there is no reliable “opened but did not click” feedback mechanism at all in modern email.

Set up a seed list of 100 email addresses you control, send through email marketing tool of choice. Open exactly 1 of those emails from a default client. See what lies their “open rate” dashboard gives you.

Above was a reply to wrong parent but cannot delete. Repost below.