Google App Engine is experiencing service outages | Hacker News Reader

Google App Engine is experiencing service outages

status.cloud.google.com

55 pointsbenpink15 comments

The error seems to be specifically around App Engine apps accessing Google APIs. I'd guess that this is a failure regarding fetching credentials, like what's described at https://developers.google.com/api-client-library/java/google....

Yes, the outage detail page mentions auth issues in its current summary: https://status.cloud.google.com/incident/appengine/15025

At least Google admits they were down. When AWS goes down, the best you can hope for is a little "i" next to one of the green checks 6 hours after the outage is over.

I'm curious how widespread it is. My app engine app is working fine, and I don't see any errors in the logs.

They estimate 2-5%

"""

We’re investigating elevated error rates for some Google Cloud Platform users. We believe these errors are affecting between 2-5 percent of Google App Engine (GAE) applications. We are working directly with the customers who are affected to restore full operation in their application as quickly as possible, and apologize for any inconvenience. We will provide another status update by 2015-12-08 12:30 US/Pacific with current details. """

We are affected. All our GCS reads/writes.

We're affected. All Cloud Storage reads/writes are down for 14 hours now :(

Edit: It's back up now.

14 hours?! Are you able to do anything within what they provide? Can you access your data in some way to move it elsewhere or something like that?

But wow... I thought 15 minutes would be a huge downtime for a service provided by Google.

Yes, this is the longest downtime I've had on App Engine for years. I've been up all night and the app engine team were up working on it as well and sending updates.

It seems that the default authentication method broke for some apps. They have different APIs to reach your data, and I tried another approach and it worked, but it's different enough, and we use Cloud Storage in too many places, that it would take a day or more to change our code to use it. I started on it anyway, in case the outage lasts longer.

We're also affected, our app runs in EU datacenters. We had auth errors on both Cloud Storage and BigQuery. BigQuery seems to work now, but not Cloud Storage. Our last outage was 2013, but this has been going on for over 12 hours..

Likewise--our recommendation engine (https://recent.io) runs on App Engine and we weren't affected, as far as I can tell.

Resolved: https://status.cloud.google.com/incident/appengine/15025

Outages are not effecting my Go Apps on the East or Central servers.

I guess Snapchat has finally brought App Engine to its knees.

Is this why okcupid is down?

Looks like it is up for me in NJ.

Reply on news.ycombinator.com