Slack is down | Hacker News Reader

Slack is down

slack.com

94 pointsnapsterbr56 comments

Open articleView on HN

Huh, Freenode is still working YEAH I WENT THERE.

And they laugh when you try to explain that decentralized services are good. :/

It's almost like the text editor wars.

That is until the next netsplit. Decentralized servers mean partition events can and do happen.

I just see opportunities for netsplit parties.

We see about 15% of Slack connections failing at https://sameroom.io

As an aside, we highly recommend having a contingency plan with another chat system on hot-standby. Here's more: https://sameroom.io/blog/when-team-chat-goes-down-are-you-re...

Would be awesome if you had some sort of status page for all chat services.

We actually do, but it's way too ugly at the moment (raw JSON).

We plan on beautifying soon.

Ironically, everyone at our company is heads down and working uninterruptedly. Productivity seems to have gone up without the interruptions!

Aside a jaunt over to HackerNews? ;)

I agree. As an aside, Slack has been getting so much usage internally (and everyone learned about @notifications) that it's turned into a massive source of interrupts. It's hard to idle in a number of channels without eventually getting @channel notifications. Perhaps it's a cultural thing here, but there's an expectation that questions are answered immediately unlike email.

I've taken to turning off Slack for periods every day to get work done.

I can't even get to status.slack.com ... I really hope that isn't on production infrastructure!

GET https://slack.com/favicon.ico 503 Service Unavailable: Back-end server is at capacity

The fact that we can't get to it means that it is, surely? :)

| "Back-end server is at capacity"

DDoS?

503 = at capacity, default response an ELB returns when all the servers in the ELB are OutOfService (or missing).

Also AWS's error code for no healthy nodes in elb.

Wow, that is a fail hosting your status page in the same datacenter/environment as production. Honestly, a bit surprised, since Slack has always seemed on top of things.

If you can't get to the status page, it's probably a good indication the service is down :)

You're missing the point. It is for communication to your customers. Give them an outline of what is happening and an estimated eta to being back up.

Usually companies do that with twitter/facebook nowadays. Status page has always been a stopgap. That's why we had http://downforeveryoneorjustme.com/

jeebus

IIRC this error message is throw when an AWS ELB is empty(in my experience). I'd imagine it could be thrown if all servers in the ELB are maxed out.

For the few people that managed to get to their status page it appears that they're also having problems there: https://pbs.twimg.com/media/CUhbPDOW4AA35Y-.png:large

ouch status page is down too

That's an interesting decimal truncation convention.

http://i.imgur.com/FiJ4G4i.png

"Let's only show precision that makes us look good."

For convenience, the numbers on their status page [1] are:

  May 2015        99.999%
  April 2015      99.9%
  March 2015      99.999%
  February 2015   99.995%
  January 2015    99.99%
  December 2014  100%
  November 2014   99.84%
  October 2014    99.9%
  September 2014  99.998%

1: https://status.slack.com/calendar

Just another reason for OSS projects to never trust closed (centralized!) protocols: no recourse when it (inevitably) hits the fan.

Except when you have to manage it yourself and something breaks or the infrastructure you have to manage needs managing.

See the comment above. If you're using Slack to avoid managing infrastructure, there are plenty of great (read: open source) alternatives.

Even if IRC isn't for you, there's XMPP, Mattermost [0], Let's Chat [1], and Zulip [2].

[0]: http://www.mattermost.org/

[1]: https://sdelements.github.io/lets-chat/

[2]: https://github.com/zulip/zulip

Mattermost team here, thanks for the mention!

Just to add, you can use IRC clients to connect to Mattermost https://twitter.com/mattermosthq/status/668898122630721538

Hey, I didn't know that - that's awesome! Keep up the good work - I'm glad to see an open source group tackling the problem of rich messaging.

I feel like this is the same sort of argument that opponents of self-driving cars will use - "We shouldn't use self driving cars, because they have problems sometimes and a driver doesn't have recourse to correct it!"

While this might be true, we are still trading more problems for fewer problems. Yes, we can't fix slack when it breaks, but it still breaks a lot less than when we were running our own internal chat client.

Except, of course, that self-driving cars do have a recourse for the driver - manual overrides. I do see your point, however, and you're correct in that running an internal chat network probably isn't the best course of action for most groups/small business. Luckily, that's what IRC and XMPP networks are for.

Manual overrides cause more problems than they solve, and probably we should get rid of them in future self-driving cars:

http://www.npr.org/sections/money/2015/07/29/427467598/episo...

This might have something to do with it? They are Inc. magazine's company of the year: http://www.inc.com/magazine/201512/jeff-bercovici/slack-comp...

What's the real world analogy to Slack being down? Locking your entire team out of the offices?

Since the start, at our company we've used Slack. We have never experienced an outage like this. Definitely a strange and uncomfortable feeling not having Slack available.

At some point Group chat is crossing into tier1 messaging infrastructure and all the sexy things that go along with it, especially when you start charging money.

I also can't connect and status slack seems down.

What I saw was their twitter status :

> We’re having sporadic connectivity issues, and we’re investigating, and working as hard as we can to resolve them. Updates will follow!

[1] : https://twitter.com/SlackHQ/status/668879517922914304

Yeah but http://chatroom.horse/ isn't.

It lacks some features until it can be a Slack replacement. What's their current valuation?

So this makes me wonder... Why hasn't Slack made a self-hosted version yet?

Having a self-hosted option would make them viable to businesses that would never consider having their team communications archived and dependent on external servers outside their control. I imagine there are plenty of those out there.

Decentralisation ftw :)

We experienced connectivity issues until a few minutes ago. Everything appears to be back to normal on our company's Slack account now. YMMV

I didn't notice an interruption on the desktop client and it's been logged in for about 7 hours now and slack.com looks fine. Interestingly enough https://status.slack.com is currently down while curling status.slack.com returns a page that says it's moved to https://status.slack.com

You mean the http redirect to https is up but the https site is down.

No I meant it returned an HTML page that said "this link as moved" with a link directing you to https://status.slack.com

Umm, yeah that is called a 302 redirect:

curl http://status.slack.com <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="https://status.slack.com">here</a>.</p> <hr> <address>Apache/2.4.7 (Ubuntu) Server at status.slack.com Port 80</address> </body></html>

The HTML parts are fallback for if your browser (or curl) in this case doesn't handle a 302

I'm getting SSL errors from httpie:

http: error: SSLError: hostname 'status.slack.com' doesn't match either of '2015-06-25.dev.slack.com', 'slack.com', 'slack-files.com', 'slack-redir.com', 'slack-redir.net', 'dev.slack.com', '.dev.slack.com', '.enterprise.dev.slack.com', 'dev.slack-files.com', 'dev.slack-redir.com', 'dev.slack-redir.net'

I can connect to my Slack team, but then "Apologies, troubles connecting to Slack" :/.

Ha. if you refresh it comes back up. Looks like only some of the nodes are down

I don't envy being their social media person right now.

Feels very DDoSe

This is why IRC is superior.

Apparently you've never experienced repeated netsplits under DDoS.

Hipchat is always down.

Or not. Is this really news?

My company is unaffected because we use MatterMost internally

Reply on news.ycombinator.com