Solving My Email Problem
blog.nawaz.org
blog.nawaz.org
For the last 20 years my rule has been "If I check my email and inbox > 1 then deal with the email right now". I usually check my email every few hours.
I'm not Mr. popular, but I'm also not anonymous. I do personal support for 30,000+ people who have taken one of my online courses so I get a fair amount of email and other things. I tend to write massive responses from scratch for a lot of them as well.
The key is not to let it build up. Then you'll have 99 problems but email won't be one of them.
Also I'm not sure if this is related but I'm one of those people who think it's unthinkably rude to ignore someone. I would never in a million years read an email and not respond to it (unless it clearly wasn't meant to be responded to). Likewise, if I see a > 1 inbox I feel compelled to handle it ASAP because I don't want to keep that person waiting.
(Of course there's exceptions, like I'm not responding to pure spam, but I do get about 50 emails a week where people ask to post a guest post on my blog and I always manually respond nicely to them with a "no thanks but good luck" type of message along with some context around their niche).
That you firmly believe this approach just means you haven't worked in a govt. dept. or any kind of position where your response represents the response of the organization (or a sub-set thereof). In such circumstances, emails that are open-ended (responses risk being misunderstood), ambiguously phrased (responses will be twisted to server the author's needs) or controversial (any response paints the organization in a negative light) are often best ignored. If your question is ambiguous, a busy boss or colleague does not owe you an answer or vice versa.
Sometimes people respond out of anger or frustration. You could respond in kind, and escalate the battle, or simply wait and let the individual calm down. Not responding is a very useful technique when conflicts arise between team members or in the management chain.
I would rather ignore an angry abusive email from an upset colleague who is having a rough time in life, than respond in kind and escalate it into an HR battle. Of course if the behaviour continues, there would be no choice, but most people are rather contrite and apologetic after a brief outburst and it's much better for everyone to just pretend that it never happened. People make mistakes after all.
I haven't worked in a govt. dept but I work in a situation where my response represents me being in business or not.
I'm dealing with 30,000+ people who email me all sorts of questions, with varying degrees of niceness. Some people are cool, others are over the top passive aggressive. I get very open ended questions, tons of "it doesn't work" with no details, etc.. My job is to guide those emails into a resolution.
If I treat anyone poorly (saying bad things or ignoring them) then I'll quickly be out of business because I don't have a monopoly. I take customer support very seriously. It's not even what I do full time either (although I'm always on call), it's just a side effect of running an online course business.
You could say instead of having 1 colleague I have 30,000+ all of which who have their own personalities and struggles, none of which were filtered through HR. I once had someone get pretty rude for no reason and then after a few emails we fixed the problem. The next morning he wrote a ~1,000 word essay to apologize because his father died recently and he drank a 12 pack of beer when he initially contacted me.
I always go down the kindness route and after answering over 5,000+ emails and questions I've honestly never had a negative experience where either party left the conversation upset.
I could turn each one into a TODO item tracked somewhere else and archive the email, then later search for the email and reply or take action. But it makes more sense to me to categorize the emails and use an organized inbox itself as my todo list.
Yes, there is some small problem of having older messages "waiting for the right time" but this is not the problem of the e-mail itself but rather mental block to dedicate 1-3h to write response (you would spend same amount of time writing regular mail).
That would only work for a while for me. 6 months later new stuff has crept in and I would have to go through another round of unsubscribing.
For spammers who don’t respect unsubscribes and you’re using Gmail, you can create a filter which immediately trashes the message based on the “from” address
That statement gets to the crux of the whole situation.
Why should I spend time tending an inbox, continually scanning for the unsubscribe link in any new email I receive? The sender put in almost no effort to send me that email, whereas I the recipient have to expend time getting off their list. I don't want to do that for the rest of my life. I'd rather just remove that email from my whitelist (one keystroke).
>For spammers who don’t respect unsubscribes and you’re using Gmail, you can create a filter which immediately trashes the message based on the “from” address
Same problem as above. I tried it and in the end found maintaining a blacklist (which is what you're essentially suggesting) burdensome. It's continual work for the rest of my life adding people to a blacklist. I inverted the whole thing. Let the sender work to get on my whitelist.
Fair point. In practice it has not been a problem. If I really want to get emails from them, I can always add them myself if they do not want to go through the trouble. On the flip side, If someone I know isn't willing to spend a minute once in their life to get access to me, it does send a strong signal on how much they value knowing me.
OK, but why wait for 6 months? Be proactive - you see something you don't want the either use unsubscribe or filter them out (if it comes from the same sender, but most of the time they give you unsubscribe link which just works…). One annoying thing with unsubscribing is that often they require you to go throu a couple of steps (including providing the e-mail itself) and confirming it so sometimes only clicking on unsubscribe doesn't work with one click.
And to make e-mail a nice tool tune what you receive - if you get gazzilion of notifications that you don't read then just unsubscribe…
I addressed this in another comment:
https://news.ycombinator.com/item?id=18102870
Consider the two methods:
1. Just have a whitelist that I've described
2. Be proactive in unsubscribing. If you buy something from someone online, be proactive at the time you buy in finding the box which says you will not receive pointless emails from them (not all vendors will have that option). Ditto if you or an app you want to run requires your email address.
My question to you: If I go the proactive route, what do I gain compared to the whitelist approach? Yes, I could be proactive and actively unsubscribe. That is time lost - even if it's not too much time for some people - what is gained by doing it?
>And to make e-mail a nice tool tune what you receive - if you get gazzilion of notifications that you don't read then just unsubscribe…
I think the difference between me and many of the commenters is this: I want my inbox (and my email usage in general) to be for personal correspondence. A few exceptions are OK, but I want the majority of the emails in my inbox to be personal emails. So it's not about a gazillion notifications. I probably get less than one personal email per day. If I get more than one non-personal email/notification/confirmation receipt etc, then my inbox is more noise than value.
Again - you make it seem like it's gigantic work to keep your inbox tidy. Last time I had to click unsubscribe was months ago and it happens like 1-2 times a year - is that too much work?
1. If someone can't be bothered to spend a minute in their whole life to be able to communicate with me, it's a pretty strong signal. I've had plenty of people who imply they want to communicate with me, but then are very poor responders to my emails. A hoop filters those people out.
2. No one has complained. If anyone is annoyed, they just ignore it and I'll whitelist them anyway when I see their email.
3. Marketers will not invent a way to circumvent it, because it's too much trouble for them to. I'm not that important.
4. Keep in mind the system has been working for over a year. This isn't a proof of concept or something in my imagination. Furthermore, if you read the other comments, people are paying money to have this feature.
5. Just because you don't get annoying emails doesn't mean others don't. Roughly half of them are unsolicited (I did not request to be on those lists). I think more likely is that the automated emails you do receive are ones you want. Whereas I don't want any in my Inbox.
My final point: Given where I am now, what possible reason would I have for switching to your workflow? Your stance comes from one who has a "default" position: the status quo. Look at it from my side - my system is working, and is very low maintenance.
Almost everyone I know who gets way too many emails and complains about 100s or 1000s of unreads has put themselves in that situation by signing up for everything. Spam is already handled well and there's just not that many sources of email if you take away newsletters and notifications.
Gmail does a lot of filtering for me though already. I write a few filter rules if I intentionally want spammy emails, e.g. watching github changes, status on shipments, etc.
My rule of thumb for managing email is thusly
- if the email is read, its done
- if theres 100 spam mail, and the top link is marked read, all ones beneath it are too spammy to delete or remove out of sheer laziness
-if the email is starred, I need to refer back to it, but its not actionable (e.g. welcome email for hosting sites)
- if the email is marked important, defer it for later
I spend very little time on email per day following these rules. I have many filter rules set up so nothing but important emails get to me in designated sections
My rule of thumb for email is thusly
- only look at email 3 or 4 times at most. Once in morning, afternoon, evening.
- send emails asap if it can be done in 5 minutes or tell receipient about a delayed response later for longer actions.
-modify subject titles to reflect status on email thread
- DRY, dont repeat yourself. Send a link instead. Use autoresponders if your on vacation etc
- every month or two, go through all your spam, unsubscribe to everything
- use autotext expanders to write email signatutes if its necessary
-use ctrl + enter to send
- limit yourself to zero to two mouse left clicks per email. If you are doing anymore, you are doing it wrong. E.g. dont organize emails its a waste of time learn to use search and make rich subject email titles.
As far as responding from aliases, I haven't figured out a way to do that in my mail client (Mail.app), though I am able to accomplish it in my provider's web interface.
this allows me tighter control on a case by case basis and I get all the benefits of Office365 business.
Way too much maintenance. This solution is easier.
No overhead of creating addresses, you have an infinite address space for your account to play with freely.
More mail providers should offer such a workflow.
With aliases, I can drop the alias and you can not easily guess my real email address.
Just depends on what you want, or what you use it for.
With Fastmail sets this up automatically if you let them handle your DNS, though you can (obviously) disable it.
You're still correct though, your solution is more secure (and also what I use). It's always possible the form you submit automatically strips the +whatever from your email silently. I believe facebook does this (at least when checking to see if the email you're using has already been registered).
I moved to wildcard forwarder on my own domain so that *@mydomain.com get forwarded to my main email account. This gives me all the tracking I want and I can ban addresses if needed.
That still gets me the zero overhead and practically infinite addresses with none of the downsides of gmail--and I can easily change my main email account as an added bonus.
For my alias management, I wrote a script to add and delete aliases and reload the MTA. Makes it super easy to do. Long time ago I would have to go through the process of creating a fake hotmail/gmail/yahoo/etc email address if I wanted to do something similar. This way is much nicer.
The email aliases will forward a set number of emails, after which they go to /dev/null. You can white-list senders who you want to continue seeing emails from.
It's all automatic so it kind of doesn't solve OP's issue, but I feel like there should be a proper way to handle the challenge-response part.
Their solution is exactly the one proposed in this article, where the sender has to solve a “challenge” to get in your mailbox.
If I think this approach is really effective, it also creates a huge pain for a lot of tools relying on emails such as WebEx invites or when you want to contact someone for sales.
As a result I think that this approach might be better if it was for instance triggered only on emails with an “Unsubscribe” link, or on emails with specific keywords.
And I am proactive. If I see an email in the quarantine folder from someone I want to be on the whitelist, I press a button and he's in. This isn't a hard line, binary, black and white situation. I can and do place people in the whitelist.
If I've requested something from someone, I don't expect them to respond to the email, and I usually reply back to them with an apology about the email. So far, no one has complained.
And I don't use captchas. There's no need for them. All they have to do is click on the link and type in their email address.
Also, keep in mind when I wrote it, I was getting 20-50 spam a day (not counting unsubscribe emails). And part of the motivation was that legitimate emails were simply getting lost (I wouldn't notice them in the sea of noise). Checking once a week would mean scanning hundreds of emails for the few legitimate ones. The challenge-response is more reliable.
(Today I get almost no spam - someone fixed a broken pipe on the Internet).
My thought is that if i asked you for an email, and I was running a system like this, i wouldn't be a dick and make you jump through that hoop. I'd proactively add your email to the whitelist before you even sent me something.
Even if they have my address, how soon is it whitelisted? Like, they run home and update their whitelist?
In at least one case, I suspect this set up is narcissistic. ‘People have to kow-tow to communicate with me.’
You shouldn't bother writing unsolicited emails with your advice to people who may not care. It's a waste of your valuable time.
I haven't used WebEx, but what is the problem exactly? Are you concerned you won't see the invite or that the sender will get an annoying automatic response?
Incidentally, this is all just a Python script. If you can construct a reasonable enough pattern for the email address, you can always have custom rules for them - it's fairly trivial. Just like I have a list where emails go straight to quarantine without producing an annoying email.
>or when you want to contact someone for sales.
My original design was to automatically whitelist anyone I send email to - in the end I didn't go that far, but it will likely alleviate this problem. My solution is mostly for my own personal email, though. If you plan to conduct a lot of business where you expect/want random people you don't know to email you, then this scheme won't work well.
>As a result I think that this approach might be better if it was for instance triggered only on emails with an “Unsubscribe” link, or on emails with specific keywords.
I just did a query. Fully one third of the quarantined emails do not have the word "unsubscribe".
Basically, WebEx sends invites from their own email address. If your customer has not white-listed the WebEx domain, they will not receive the WebEx invitation. The only solution we found for our sales team is to "double" the invitation with a manual email sent separately, with the link to the WebEx invitation...
Also, I just realized that I was not very clear in my comment: my company does NOT use MailInBlack :). However, a lot of our customers do, and this has been a nightmare for our WebEx invitations process.
That's rather obvious. If they want to receive emails from WebEx, they should whitelist it. Keep in mind that the quarantined emails are like emails in any folder. You can still check them to see if there are important emails there.
Now I don't know much about WebEx, but ... are you sending WebEx invites to people who are not expecting an invite? If so, I would say my filter is doing its job! That's exactly the type of email I'm trying to cut down on. If, OTOH, I am expecting an invite from you, I will check the quarantine folder for it. And as I said earlier, if WebEx becomes big enough that I expect many people will use it to contact me, I'll just whitelist and put it in the low priority folder.
Fundamentally, the problem is that we've overloaded emails. Emails (for most people) were a means of communicating between individuals. Then people started using it as a TODO list. Then as an advertising platform. Then as a way to manage receipts. Then as a calendar system. And on it goes. One of my goals is to separate the personal emails from everything else. I may still use it for other things, and set up scripts to handle those other things, but I need a way to separate out personal emails from everything else.
Now if WebEx also starts sending me unimportant emails, they're out of the whitelist. Kind of like LinkedIn. It uses messages-noreply@linkedin.com for all its emails - whether it is to notify me that someone sent me a message or to let me know that "Hey, if you're willing to become a Premium member, everyone will want to hire you." - They are not in my whitelist.
And BTW, there is only one customer: Me :-)
Did you know about TMDA before you wrote it?
Also, almost all their links (Documentation, etc) are not working - I think they're having server issues.
Their site is not shinny but the product does the job. https://www.mailinblack.com/en/
Laudable effort, but it’s not a solution.
>What happens when an automated system sends you a message?
I'm not sure I see the problem. I don't want automated emails in my Inbox. Period.
For the few ones that you'd think I want (e.g. overdue notices), they are in my "other" list - those that don't go into either my inbox or my quarantine - so they don't get an autoresponse email. But frankly, there are very few automated emails that I want.
As an example, I've set up Zillow alerts that tell me if a house in my neighborhood is on sale, or has been sold. In one sense, I do want these emails, but I simply don't want them in my inbox. They remain quarantined, and I'll see them whenever I check my quarantine pile. Most of the sites I intentionally signed up for because they occasionally will send me something useful - they all still go to my quarantine pile. I don't need them in my inbox intermingling with more important emails.
>How do you handle the backscatter problem (ie when spammers spoof the sender address causing your helpful challenge messages to be sent to someone random who then complains..).
If the spoofed email is in my whitelist, they don't get an email. If not, what exactly is the problem? That some random stranger got an email from me? That's just spam for them - I doubt they'll even notice. At best, they may realize someone is using their email address as a From in a spam email.
At least in the 1+ year of running this, none of these has been problems.
Anyway if it works for you _and_ is simple to put in place and use (as you've shown), then kudos to you!
First, I have a tendency to keep emails in my inbox as well, thinking I will do something with them at some point. I can't say I've ever waited a year to respond; however, I have probably waited several months before deleting (archiving, since I use Gmail) an email. This often comes after realizing that I am not, despite my earlier assessment, going to do anything about this particular email. Second, the endless notifications are abhorrent.
Today, I finally got around to adding some little used account information into my KeePass database and archived two emails. Then I realized I wasn't going to do anything with the other 20 emails in my inbox, so I archived them, too. Then I turned off notifications on my phone. I turned off Outlook's toaster at work years ago after I was frustrated with the amount of useless junk my colleagues or other work groups keep sending me. Unable to remove myself, I just decided to deal with the junk and all the other emails I get on my time, not anyone else's. I took that practice to my personal accounts, after deciding that if I won't waste my time even when I'm getting paid, I should definitely not waste my time when I'm not.
Now if I could only talk my employer into letting me disable my email account when I go on vacation, a la Daimler: http://time.com/3116424/daimler-vacation-email-out-of-office....
Every time I got a spam leaked (into unsorted) and I am annoyed or I want certain mail go to certain inbox, I simply edit my scripts (which I know exactly where and how). It is easier than -- e.g. -- reading procmail's man page.
Specific, simple, personalized approach is surprisingly effective than any generalized approach. The latter is a hard problem, but the former often isn't. The former I can choose my own compromises.
NOT counting the spam folder i have 52,541 unread emails.
I am not going through all of that to click a bajillion unsubscribes considering i never actually chose to receive email from 99% of the people / companies sending me email.
I'd forgotten about the whitelist strategy. I'd written it off as obnoxious to the sender, but as i am reminded of this years later i'm thinking it's more obnoxious for me to never respond to people, partially because i miss their emails, and partially i don't bother looking most of the time because there's too damn much to deal with.
This is exactly my perspective as well - I even said in another comment "I don't want to do this for the rest of my life". Reading the comments, it is clear not everyone has assigned the same roles to email as I have (which is understandable). For some people, opt in as a default seems to be acceptable.
I don't know - perhaps because I had email for some years without getting all these emails, I may simply have a different reference point than many who grew up at a time when communicating individually over email was the rare use case (as opposed to being notified of sales, meetings, etc).
(Note: Definitely not complaining or being sarcastic here - to each their own).
I very rarely check my spam box, only when I'm job hunting as that's very critical I don't miss anything.
- a good taxonomy on a maildir in my home and on my IMAP (thanks mbsync+cron+a small script to watch inbox with IMAP Idle)
- a good automatic filtering/refiling (imapfilter, for now)
- a good spam blocker (spamassassin)
- a good MUA which support both physical taxonomy and a search-based one
The workflow it's roughly simple: I have some saved search in notmuch-emacs mostly 'unread', 'important', 'inbox', 'live', 'todo'. When I have little or no time I look only at unread+important, when I have time I go through the rest of unread. Live stuff are thing in progress that I have to watch but no action or todo, todo are...todo's...
It didn’t worked. People receiving the quarantine mails often misunderstood it or dismissed it without reading it.
We tried with CAPTCHA, without, nothing worked.
Our users had to spend a lot of time checking for false positives.
In the end we gave up.
Oh, and if you want to do something like that, add a reasonable delay for your response as some Spam list check for automatic responses and consider them as SPAM and will ironically block you for that ! Been there...
I switched to Rspamd and haven’t looked back. It’s much better.
For single user mail folder/server and local use, it's very simple to set up, compared to anything server side. You basically use 3 commands after initial training and don't need to worry about anything else.
I don't want to keep an e-mail archive server side (I use pop3), and that would include a spam filter's database, there are not many other filtering options with that requirements.
Anyway, I agree it would not make a good first impression without training data, at all. Though if you train it with what you actually receive, it's quite awesome. I get > 99% accuracy for both false positives/negatives.
It's good to keep spam for training.
What is generally left in my inbox is stuff directly mailed to me that deserves attention. 90% of the stuff in the unsubscribe folder get deleted without looking beyond the subject line.
But my reason for such slow response, I believe, is a little bit different than most. I have social anxiety, and reading email is extremely painful to me. Not replying, but reading. I almost never open my inbox. The last email I sent was six months ago.
Sigh. Anyone with similar issues?
I may end up using a separate domain for those emails. I'm pretty sure that's a weak solution.
It sounds like you've invented a captcha for email. If that ever took off and became widely used, it would be rather easy for the spammers to write a script to comply with the scheme. Followed of course by the usual escalation to adding an image-based captcha, and then efforts to beat the captcha, and so on.
As an aside, you could do it entirely in email as well by sending a response that requires a specific reply to join the whitelist. That is you could eliminate the webserver and the mildly-questionable-looking URL.
Large-scale email in its current form sucks, even with the Office 365 ATP tools or similar equivalents.
I've always gotten more spam in Hotmail than in Gmail (not a lot more). But I've had more false positives in Gmail than in Hotmail. Always seems like there's a tradeoff.
https://freedom-to-tinker.com/2003/05/19/challenging-respons...
Now I'll grant there is a critical mass beyond which CR won't work. Looking at the comments here, the likelihood of hitting that critical mass is almost zero. Most people don't like the idea. So CR will continue to work.
So what's the point? The whole exercise is intended to cut down on looking through non-personal./junk email, but you have to look through non-personal/junk email daily anyway.
I use Google Tasks as a workaround (link a task to an email conversation) but it's still clunkier than Inbox.
I don’t have an email problem anymore. I have a WhatsApp, Slack, Facebook Messenger, SMS, and Skype problem now.
"Jeff is contacting you" over what? I don't care, I have a message from Jeff, and I can respond to Jeff over any protocol I choose, and the shell will show me a unified history of my IMs, chats, phonecalls, SMSes, Deathmatches, emails, chess moves, teledildonics sessions, whatever with Jeff.
https://www.youtube.com/watch?v=PBWRdtC9Cn8&feature=youtu.be...
Stuff like Amazon.com emails are whitelisted. They go into the "low priority" folder, not my inbox. For one-off vendors, I simply "save" their emails without adding them to the whitelist. It's easier to understand if you've used notmuch. I simply remove the quarantine tag from them and they're stored in the email database. One keystroke.
I'm not any different from you. If I purchase anything on the Internet and get a confirmation email, I aim to keep it.
I still check the quarantine folder from time to time.
>confirmations of online accounts
I'm not sure I see the problem. I still can see the quarantined emails. If I'm expecting such an email, I check it within a few minutes. I click on the link to confirm my email address - I merely do not add the sender to the whitelist.
>all sorts of other stuff where I am interacting with, or storing information from an automated agent.
An example would help. Some use cases from me:
- Calendar reminder emails - Whitelisted: They go to the low priority folder (as counterintuitive as the phrase sounds). They do not go into my inbox.
- Library overdue emails - same as above
- Emails from my medical provider (e.g. "new test results") - same as above
If you get automated emails for which you have automated actions, those can go right into the Python script and handled before they are quarantined.
I completely misunderstood that the quarantine is just a folder next to the inbox. It's the obvious solution, from both a technical and usability point of view, and takes care of all my points.
Thanks for clearing it up.
So, how about a proof of work system like blockchains? Like, if you've never sent me email before, my server returns some work to complete. If I've ever sent you an email, or not marked you mail as spam etc, your MTA won't be challenged etc.