Hacker News Reader

Hacker News

Comment by tony | Hacker News Reader

Parent Full thread

tony

The toothpaste is out of the tube - notifications are going to be scrutinized more in the future, from this point on.

I got the email. And about 20+ responses to that email.

Every person who replies to that issue triggers another 400k emails. Personally, my email client is crashing.

EpicGames, as a GitHub org is an outlier, it's basically an SSO for Unreal Engine. I forgot I was even a member of it.

View on HN

Yeah I’m sure they’ll institute some kind of control to make this sort of thing harder, but honestly I’ve never understood why people get so worked up about this sort of thing. It makes me chuckle. The person looks like a dummy or a shit-stirrer and a lot of people have to delete HUNDREDS (oh my) of messages and have LITERAL MINUTES of their time wasted. The megacorp I work for wastes more of my time with silly self-congratulatory org-wide emails about business deals and fake benefits like seminars on retirement planning for dummies.

I love me a good bedlam drama. One of the commenters on the PR had the best take: “I just wasted 2 minutes of my life I'll never get back.” The ones with the scorched earth PUNISH HIM attitude need to chill the hell out.

There’s no real way to explain why it’s annoying. You just have to accept that other people have different personalities from yours.

And those personalities want to severely punish someone over a minor transgression. Different strokes for different folks, I guess you could say.

>You just have to accept that other people have different personalities from yours.

Yeah, being drama queens.

> The ones with the scorched earth PUNISH HIM attitude need to chill the hell out.

Doesn't that just add to the comedy?

Yeah. Also their real names and them being immature and getting angry also went out to 400k people and is now effectively part of history. That's probably even worse than being the kid who did this, what if a future employer comes across this?

friendly reminder: 400k is an apertif for a competently configured production postfix server. its about 14 seconds of mail, and about 8 seconds optimized at hw and filesystem level.

the real issue here is shitty projects from shitty companies.

A lot of developers seem to underestimate how fast well-written software can process things...

Where I work, we send over a billion emails a month (blog subscriptions) and 400k is not even a blip.

Yes but that’s on top of what GitHub is already sending, plus it must be multiplied by the number of comments left on top of that. It feels like 35 straight minutes of 100% usage isn’t great on any system. It presumably sent 61 million emails.

> Yes but that’s on top of what GitHub is already sending

When you send at the volume someone like GitHub sends, you will always see peaks and valleys in your sending patterns that are much larger than 400k. It might cause an issue if they were already under peak load, but even then it would just take a bit longer.

GitHub can afford more than one computer to send their emails.

Yep, the mail bodies would be cached out while the rest is just the addresses in queues.

oh.

> I got the email. And about 20+ responses to that email.

> Every person who replies to that issue triggers another 400k emails. Personally, my email client is crashing.

Your email client only received 20 messages; why is it crashing? The very long To: header?

Why would it be a very long To header? It’s not like the email headers include all 400k emails in one go.

That would be a massive privacy breach, and people here would be making a lot more noise about such a large email database being leaked.

I agree, but couldn't think of another explanation related to that email.

Drunk Microsoft programmer who developed Outlook

Attempted to reach the Ballmer peak but went too far

It's a personal testimony. Immaterial to the main point of the post.

There was a thundering herd e-mail at Amazon about 10 years ago that I’ve heard stories about. It went on for days. There’s a funny internal talk with lots of data about it, maybe it’s on YouTube by now…

Looking around, everyone has a story like this, so I don't want to just pile on with my similar experience at Verizon. But what stood out at me there was the low quality responses from low-level managers in far parts of the world, demanding, by the power vested in them, to be taken off the email chain... you know... IMMEDIATELY!

Yeah that is one thing that I remember from one or two that I was copied into in roughly 2000s with more than 200,000 CCed. Mini-kings and also people with enough technical skills to know better keeping the threads alive and flooding the Exchange servers woldwide for days.

This happened at all my previous employers at one point or another. Most famously a thread about unsubscribing from a mailing list that nobody really knew about but had everyone in the company on it. For weeks there'd be some random field sales guy or a marketing person in random parts of the world replying all.

Something similar happened at Cisco something like maybe 5 years ago? Someone sent the Bay Area employees or whatever list (probably at least some tens of thousands of employees) an ask for a cook/chef they could hire to make meals for their family, IIRC. I think the reply-all's happened all week.

That was sometime in 2015-2016. People still post recipes in big reply-all chains or when cisco-flame gets too heated.

Dang that really takes me back.

MS in 2005 as well.

And Apple in 1991 [1]. It's a very fun story actually, good read if you're in the business of writing "reliable" or "recoverable" software (aka pretty much everyone). A bad design choice in sendmail caused a cascading explosion of emails. I also highly recommend reading the rest of the book (?), the Unix Hater's Handbook is a wonderful bit of history and discussion of the design issues of the NIXs we all know and love/hate today.

[1] https://web.mit.edu/~simsong/www/ugh.pdf page 85

"Bad design choice" described basically every design choice involved in email.

Compaq in the mid 90s on banyan vines a friend leaving the company sent a company wide email listing his house. It wasn't supposed to be possible to email *

When i worked at the NIH, a couple times an entire institute was accidentally cc'd instead of bcc'd on an email, and for weeks after the chain would continue, consisting only of people writing "please stop responding to this chain". I don't know what you do for that...

Where I work we create a policy that drops the messages.

To: soandso@domain.tld Subject: blah Action: block

The people responding with “please stop” should bcc the institute so that if someone replies to them the institute won’t be copied in?

Mute the thread in your email app?

Would that happen to be the "wallet" incident? It was slightly before my time, but I also heard legends of it. During my time there, any email thread that looked headed for another reply-all storm had people replying-all to it with simply the word "wallet", apparently in an attempt to deliberately cause chaos.

Wallet?

> notifications are going to be scrutinized more in the future, from this point on.

This already happened with github & epic & unreal when it first did this organisation setup. So, given no solution appeared after exact same incident, I wouldn't hold my breath

It's an effective setup to make people feel included or part of Epic in some way. (Adding everyone who wants access to the organisation.)

I'm up to about 156 replies between two emails now. My phone and smart watch were buzzing like crazy earlier for each notification.

The real question is why on earth do you have notifications enabled for all emails?

Or github notifications at that. It's already very noisy.

That's where it ended.

It must be hard for GitHub servers, but which e-mail client is crashing from about 100 short, text-only messages received in an hour?

I doubt it but I wonder if OP is lying and didn't realize there would only be a hundred or two emails

Can you restate.

Most probably Outlook! Imho the worst e-mail client on earth with a massive base of absolutely blind fanboys in almost every company...

Which Outlook? There are several different email clients with that name.

Outlook 2003?

my outlook 365 (or whatever it's called this week) at work can barely handle moving 100 without locking up

Irrelevant to the topic.

For me, it is Thunderbird, but only because Windows Defender scans all incoming emails AFAIK.

Your email client is crashing because it can't handle a hundred messages in an hour, that's a little less than 2 messages per minute?

Throw that thing in the bin. A human printing emails and placing them on your desk could handle that workload.

> A human printing emails and placing them on your desk could handle that workload.

Best thing I've read today only after the email story.

That'd be a pretty drab day.

Reply on news.ycombinator.com