Comment by valerij | Hacker News Reader

Comment by valerij | Hacker News Reader

reminds me of that one time a genius at IBM created an `ibm-global-announcements` channel and force-invited 200'000 people in it, and then some guy `@channel`d and all ibm's slack workspaces were down for 30 minutes

https://status.slack.com/ibm/2018-03/f01d4c22cd953dd7

https://i.imgur.com/Rk6Kdgp.png

EDIT:

also that channel made using slack impossible for mac book air users, they had around 80% cpu usage for slack. so basically entire marketing and PM part was unable to work that day. developes machines were wasting around 20% on slack.

after people started complaining in that channel, posting in it was limited to admins only, but they didn't lock commenting. so, for approx 6 hours all of IBM was posting memes in ＴＨＥＴＨＲＥＡＤ as we dubbed it, and @mentioning the genius who created that channel. next day the channel was nuked, not even an archive preserved.

some guy calculated that the entire affair, considering electricity prices, a 20% decrease in developers productivity, and so on resulted in IBM loosing several millions with that stunt

fun times

p.s. shout out to Martinj for that spicy jeff-coffee-mug meme

> a genius at IBM

Are you saying it is the user's fault where it is clearly Slack's inadequate implementation that is the problem?

i stand by my opinion that "lets create one chat room and invite hundreds of thousands of people into it" is a bad idea regardless of what technology implements the chat room

I respect your opinion, because even if an application could handle 200k users at once was able to be run on a MBP without serious issue, it's still gauche to forcibly invite 200k people in your company to a chat.

Slack is designed for business chat, and many businesses have thousands or more employees. Slack should be able to handle any features it provides, and @everyone is a feature. I don't know how they've implemented things or what the specific issue is, but it should at least have blocks in place that prevent stupid things. Maybe channel sizes should be limited to 1000 people or whatever their testing shows it can handle. Idk.

Software designed for large numbers of people shouldn't go down because one of those people does something stupid.

If you want to tell everyone in your company something, you could email them I guess. Is it the fact that it's a "chat room" which means you expect it not to work? What's best, here? SMS? Email? Putting something on the website and making everyone check the website once a day at noon? Tell one in ten people and get them to "pass it on"?

Is this a serious question? Companies have been using email for internal announcements for decades.

Yes, and he's making the point that it manages to work. And that perhaps your expectation should be the same for a chat client as for an email client. Which may or may not be a valid point.

And without proper mailing-list-moderation, that too can cause email storms. No one blames "the genius who decided to put everyone in one mailing list" when that happens.

The real fail here isn't putting everyone in one channel. The real fail is not using/providing proper admin tools

I'm going to guess you could also break IBM's phone system by attempting to start a conference call with 200K people on it, and destroy their auditorium by putting too many people on the stage.

We definitely blame the genius that didn't BCC an email with hundreds of individual emails (when a mailing list isn't used)

Although usually only a limited number of people can post to an "announce-list" that goes to everyone in the company. (And if there are company-wide lists that anyone can post to, most users filter those to a folder. I suspect such lists don't scale past a certain point either.)

Most companies today have a decent intranet site for these things.

I love that companies will make it hard to expense a $50 meal but they will gladly let you book an hour meeting with 20 high-paid developers or even let you directly ping 200,000 people which must cost the company many thousands of dollars.

Well, you know what happens when you email @yourcompany.com? Is it a protocol problem? Implementation problem? User problem?

It's a lot of problems at once...

There should be a way to turn off @here and @channel. It should be a user-level setting, a channel-level setting, and an organization-level setting. Even if my org or channel doesn't opt out, I should be able to.

I hate that it can't be controlled per channel. There are many channels where it's useful and desirable, so our org doesn't disable it globally. This creates situations where new people @here and @channel not knowing any better, which leads to hurt feelings all around.

Some of this you can already do. Users can opt out of @here and @channel for any given channel. I know that orgs can restrict usage of @everyone, I assume they can do the same for @here/@channel but I'm not sure.

I know you can in Discord, I'd be shocked if Slack doesn't let you do that. checks Settings

Yes, you can mute a channel and suppress @everyone and @here if you want. Still friggin nuts that a chat service can bring a modern machine to it's knees.

Ah I knew you could mute it but muting a channel does not suppress @here. I did more exploring and found it's a multi-step process to suppress @here vs muting, but it is indeed possible.

Learn something new every day.

Or at least a way to force users to pay a fee whenever they use those features.

It would be interesting to have a karma function whereby it was based on your use of @channel or @everyone - or something similar to how HN used to require 500 karma to be able to downvote...

People at my company use @channel constantly for the most trivial of things its a major pet peeve for me

My work has a 1000+ person channel for ad-hoc support questions that aren't critical and could be answered by anyone. If it's critical, our support team has an SLA, but Slack doesn't.

There was one specific person who would ask a question, expect an immediate response, and if they didn't get the response they would @here. They were instructed not to do that because it pings over 1000 people, so the next time they asked twice in an hour and when they didn't get a response they said "I hate to do this but I really need a response @here". They were kicked from the channel. Really unfortunate since it's a very useful channel to be in, but @here and @channel is just so disruptive in big channels.

It should just be turned off for large channels. Pop an error message and direct users who need it to a support article, if they follow up real support people then see how many people complain.

I remember that day fun times. I just shut off Slack and got back to work was nice not being pinged constantly.

I still hate that anyone can add anyone to a channel - it's not an "invite", it's just global "add to room" privileges.

In general I find Slack not very concerned with your own ability to control your experience (no ignore user feature, can be added to rooms, can be added to rooms that then get pinged with @channel and your notifications start blowing up, etc).

This may be 'expected' in a small company where of course everyone should talk to everyone, but for larger companies and the plethora of various open-source or interest-based communities now using Slack, it's not so good.

Even worse than the channel mechanic is the thread mechanic - anyone can turn your comment to a thread, and you cannot mute threads by default. Even worse, updates to threads suppress channel unread notifications, causing you to miss updates.

Manually having to follow threads and unfollow threads is a nightmare compared to the one time configure experience of channels, especially since you cannot just mute all threads, and is easily the worst part of Slack to me.

I'm more surprised that the IBM server has 200k users?

According to Wikipedia, IBM has 380k employees. Not that surprising that half of the use the company Slack.

This is not old news. There is still an IBM slack channel called #general with 30k+ people in it. And people regularly arrive in it to say "hi". No useful information is shared.

IRC works when you do that still :)

OT, but:

> 200'000

Interesting. Why an apostrophe?

i've picked up this kind of thousand separation at some point to make large numbers scannable. c.f.

    1600000000
    1'600'000'000

separating at 10³ is due to counting customs of the region i live in.

using an apostrophe `U+0027` appears to be the least wrong use of possible glyphs accessible on my keyboard layout

also c++14 does it this way

It's common practice to use commas for this in anglophone countries, but it collisions with french decimal separator; so it's a reasonable choice.

Distributed Denial Of...Slack?

Reply on news.ycombinator.com