Hacker News Reader

Comment by xbryanx | Hacker News Reader

I'm getting the same outage in us-west-2 right now.

The dashboard doesn't load, nor does content using the generic S3 url [1], but we're in us-west-2 and it works fine if you use the region specific URL [2]. In practice this means our site on S3/Cloudfront is unaffected.

[1]: https://s3.amazonaws.com/restocks.io/robots.txt

[2]: https://s3-us-west-2.amazonaws.com/restocks.io/robots.txt

Good catch. My bet is that because s3.amazonaws.com originally referred to the only region (us-east-1) the service that resolves the bucket region automatically is really hosted in us-east-1. I think AWS recommends using the region in the URL for that reason, however that is easier said than done I think. I would bet that a few of Amazon's services use the short version internally and are having issues because of it.

Seeing it in eu-west-1 as well. Even the dashboard won't load. Shame on AWS for still reporting this as up; what use is a Personal Health Dashboard if it's to AWS's advantage not to report issues?

Now it's in the PHD, backdated to 11:37:00 UTC-6. How could it take an hour to even admit that an issue exists? We have alerts set on this but they're useless when this late.

Same here, and it's 100% consistent, not 'increased error rates' but actually just fully down. I'd just stop working but I have a demo this afternoon... the downsides of serverless/cloud architectures, I guess.

Well what if you'd hosted it on your hard drive and it crashed? It seems like the probability of either is similar nowadays.

The difference there is you can potentially do something about it, vs having to wait on an upstream provider to fix an issue for everybody.

"you can potentially do something about it" vs. "you have to do something about it"

Perspective is everything.

Grab different machine, git clone your repo, good to go.

What's the odds of the server with your repo and your own hard drive crashing at the same time?

Strangely, your comment made me read this entire post about working out probabilities.. http://www.statisticshowto.com/how-to-find-the-probability-o...

Quite interesting really!

If we assume that the events are largely uncorrelated+ then we are multiplying the probabilities and our chance of wipe out are far lower.

+I would suggest that for situations where the probability of my machine and github's/bitbucket's servers being down due to the same event would be events of such magnitude that I would not be worried about my project anymore being more focused on basic survival...

Heh that "increased error rates" got a chuckle out of me, I guess 100% is technically an increase.

Our services in us-west-2 have been up the whole time.

I think the problem is globally accessible APIs are impacted. As others have noted, if you can use region/AZ-specific hostnames to connect, you can get though to S3.

CloudFront is faithfully serving up our existing files even from buckets in US-East.

S3 bucket creation was down in us-west-2, because it relied on us-east-1 (I expect that dependency will get fixed after this), but all S3 operations should have continued to function in us-west-2, other than cross-region replication from us-east-1.

IIRC the console for S3 is global and not region specific even though buckets are.

Also, cross-region replication is a new-ish thing: https://aws.amazon.com/blogs/aws/new-cross-region-replicatio...

Same outage in ca-central-1

I can confirm this as well.

Huh, I'm not seeing it on my us-west-2 services. Interesting.

Reply on news.ycombinator.com