Hacker News Reader

When someone reports to me "The website is down" I roll my eyes and immediately seek more information. I wrote up information below, imagining that one day I'd share it with someone reporting that "The website is down". Of course, I probably never will because I try not to be a dick. But I'll share it here:

   FYI, "down" has a lot of meanings which most people don't differentiate between when reporting the problem:
    1) Site can't be connected to at all (connection timeout), due to one of the following:
     a) DNS doesn't resolve to any IP
     b) DNS resolves to the wrong IP
     c) DNS resolves to the right IP, but the server is powered off or otherwise unreachable over the network
    2) Site responds with a connection reset
    3) Site responds with "connection refused"
    4) Site responds, but is slow to load
    5) Site responds but has invalid or misconfigured TLS settings, resulting in the browser showing a security error.
    6) Site responds with a server error (HTTP code in the 5xx range)
    7) Site responds with a 404 Object Not Found error
    8) Site responds with a redirect loop, causing the browser to give a "too many redirects" error page
    9) Site responds with little or no content. (Empty page)
    10) Site responds with the right content, but it is abnormal:
     a) Missing Images
     b) Odd formatting (i.e. missing CSS)
     c) Some dynamic behavior doesn't work (i.e missing JavaScript)
    11) Site responds with other, unexpected content.

Saying a website is "down" is WAY too vague to be useful. Heck, at my old job I was told "Our dev site is down" and I'd have to ask "Which dev site?!" because we had several sites. Ugh.

I totally feel your pain.

This list of items is called debugging, and it's probably your job and not the non technical user who reported it to you ;)

I may have gone a bit into the weeds with the some possible causes, but nearly each point has a distinct difference that can be seen. Sure, the user may look at a website/browser error and not have any idea what it means, but it doesn't mean they can't relay the message.

For example, "The website is down" vs "When I go to the website, I see the message 'This site can't be reached. some-invalid-website.com's server IP address could not be found.'"

“Go and see” is great. I have been in a lot of discussions where people talked about something for hours that they had never seen first hand. It’s so valuable to see directly how your users are using your product or to see people’s workplace. With offshore people we had problems lingering for weeks until somebody went there, saw what they were actually doing and what equipment they had, and solved the problem quickly. Same for setting up equipment at customer sites. Struggle for months because the local sales guy doesn’t give you the correct information or go there and solve the problem in an hour with their IT guys.

All the data we are collecting is useful but seeing things directly is extremely useful. Seems the modern workplace is often becoming very abstract where everything is compartmentalized and people see only their little box.

One of the nice things about where I work is that QA is a very short distance down the hall. When they tell me "It's doing X", I can walk over and say "Show me". And then I see that they didn't mention that it was in mode Y when they made it happen, and they did step Z on the way to get there. They may not have thought that those pieces mattered, but to the developer, sometimes they do.

“And then I see that they didn't mention that it was in mode Y when they made it happen, and they did step Z on the way to get there. They may not have thought that those pieces mattered, but to the developer, sometimes they do.”

Exactly. You take things for granted and suddenly all your analysis work is based on incomplete or wrong assumptions .

Because of this, screen-sharing capabilities are critical to resolving computer-related problems that people report.

I once got a report from a coworker that he couldn't import a text-based configuration file into an application. I was unable to reproduce the problem. It wasn't until I had a screen-sharing session with him that I saw the problem. The configuration file extension wasn't correct. It looked right at first, but then I realized that it had an extra .txt at the end of it that was being hidden by Windows. The application saw the .txt at the end of the filename and rejected the file.

Nothing wastes time like emailing/texting/chatting back and forth when a voice call and screen share are available.

Oh, and calibrating the level of detail in technical conversation is very hard too, because it can lead to people feeling like they are being talked down to, or it can lead to people assuming the other person knows more than they do. It's a Hard Problem.

In the end, I think, people want their problem fixed ASAP. To that end, I make zero assumptions about what others know, especially over asynchronous communication (and it's frustrating when you're working with someone halfway around the world and only get to leave them one email per day, for example). I'm aware that it might come off a certain way, but in all honesty, more than feelings, I'm interested in getting issues fixed as quickly as possible and giving people all the info, techniques, tools, etc. they could possibly need to fix their problem now, fix a future problem, and maybe prevent this problem from happening again. I've gotten negative feedback once, and when I explained why, it seemed like they understood and there were no hard feelings.

I think of it like this, if the shoe was on the other foot, it's how I would want it to be handled. My feelings aren't hurt if someone is working from a script or dumps a set of five steps at me, four of which I already know, etc.

Demoing fixes (or research) is pretty key in the SRE world. Even SRE's across different domains (SRE-SE <> SRE-SWE <> SRE-DB) will inadvertently question each other. Especially for the fact that a lot of problems you're solving look complex through the lens of abstraction, but once you get them to the primitives they're quite simple.

>The System is Down

Strongbad's first techno song!

https://youtu.be/O4gqsuww6lw

Mind sharing the said document, or would that be too much of a favour?

See also justin_oaks sibling comment: https://news.ycombinator.com/item?id=30211985

And "How to ask questions the smart way": http://www.catb.org/~esr/faqs/smart-questions.html

The important part is tailoring it to your environment.

Also ask yourself if your system makes issues observable and testable. If you're spending a lot of resources on support or testing, those may be features worth adding. This is hard to quantify and defend to management, so sometimes you may just wind up working on it for your own sanity.

---

What to include in your question:

ENVIRONMENT

EXCEPTION

WHAT YOU'VE DONE to localize or investigate the problem

SERVICE NAME

URL

Before you ask Is X Down? Please take a moment to ask yourself these questions:

Is the service up? (Check Cloud Provider X, Cloud Provider Y)

Is the server up?

Which environment am I asking about?

Have I checked the logs?

Have you googled the exception?

Have you looked for code related to the exception in the codebase?

BAD: The service is down.

GOOD: When I run service ms-random locally, I get the following error.