Hacker News Reader

Comment by astonex | Hacker News Reader

What the hell are npm doing that this caused a prod outage.

Either way, really poor from Smyte. No reason to immediately turn off access

If I was including an anti spam/security service as part of my pipeline I probably would want my system to stop if it went down, rather than just skipping the protection. Especially as cutting me off from said service is a possible attack option for a bad actor.

Seriously. Shutting down a security product without warning is just reckless...

I could imagine (just guessing): submit new repo -> scan with Smyte -> on success, publish repo. Maybe not handling an error properly and skipping Smyte.

This problem creeps up with external mfa/2 factor auth APIs that go down, bringing down the main login. Some choose to skip mfa if down, some don’t

It's the not handling an API failure correctly that surprises me. That seems like it'd be quite common

I mean, sometimes handling an API failure correctly is failing over. If an API call actually matters, then when the API stops responding, throw an error. Even if you're probably catching that and giving out an error message, I'd still call that a production outage.

We have a differing opinions on what outage means then. Catching the error and showing it to the user is obviously the right thing to do, but not what I'd call an outage.

I can't say I've ever met a developer who thinks that a 100% error rate to valid requests isn't an outage. Not sure why you have such a strong view of your semantics.

Sure it is, if it affects all users at once, and makes your system unusable. If I dial a phone number and get a busy signal, that's an "error" of sorts. If every Verizon user gets that busy tone on every call, that's an outage.

This is a spam filter, so failing closed seems reasonable. (Failing open is also reasonable too.)

There are lots of potential security issues with npm (compromised accounts, spam packages, etc). I think failing closed is the right thing to do for them. It's a temporary annoyance and loss of productivity for a day - a good time to put your feet up and relax. Failing open could lead to a costly security breach that affects many people, and could lead to npm's demise.

npm gets a lot of flak on HN, but this seems fine (or rather, it should have been fine) given that they and others apparently had multi-year contracts with Smyte. What kind of situation would you be in if Github went away, for comparison?

This seems like a clear breach of contract. I can’t imagine how anybody thought it was a good idea. Maybe it was a mistake, although that wouldn’t be a whole lot better.

Reply on news.ycombinator.com