Gmail and Google Drive Outage
google.com
google.com
My pet theory wasn't too popular but I'm going to stick with it :)
I wouldn't doubt that Google Classroom and other systems that use Google's SSO will be under strain from millions of students.
And similar concepts of maintaining by sysadmins are practiced.
Press and hold the F5 key on your keyboard for 2 minutes while on gmail.com. You will get a "service unavailable" error. About 500 other people whose data happens to be cohosted with you will also get the same error, and all of you will be unable to send or receive email, even by IMAP, for about 10 mins while your particular corner of the data store is restarted and the data integrity checked.
That doesn't happen on Google.com
Don't believe me how slow it is on a regular instance? Try going to "All mail", selecting all of your emails, and applying a label to them all. In my experience, it can only label about 50 mails per second, so it can take hours to do them all. It will keep going if you quit the browser, but will stop if the gmail devs do a software update, which they seem to do on usually tuesdays, but never fridays or the weekends.
I’m sure they have machines that are only used to serve G Suite and Google One customers, and maybe some other VIPs, but regular heavy users? It sounds like an urban legend to me.
for Gmail, you have to log in. It has to know it is you. It has to know that everything it is serving you is only for you. It has to retain copies of documents that -- as far as it knows -- are unique to you or whomever you share with. It has to do all this while keeping that information safe from other people who might want to take a look at it.
And that's just the items I can think of in real time while typing.
Was that this week?
And if this happened last year too, you would think this would be on top of the list of things to watch for next year and add capacity for. Amazon and Walmart start planning and drilling now for their holiday season.
A lot of people would immediately dismiss it because Google has the resources to scale up. But having resources doesn't guarantee someone actually turns the knob that increases the number of instances. (Whether automatic or manual, the adjustment could be too slow to match an unanticipated spike in demand.)
But there's another reason I don't think that's the explanation. Gmail has 1.5 billion active users[1]. Millions of students logging in at the same time sounds like a lot, but if Gmail has 100 million more active users today than yesterday, that's not even a 10% increase!
---
[1] Source: https://en.wikipedia.org/wiki/Gmail
IMO it's not really about having the resources to scale, but the unpredictable emergent behaviours which can happen when the load profile suddenly changes
"We are experiencing an issue with Google Kubernetes Engine (GKE) clusters using node auto-provisioning becoming stuck during node version upgrades. Node auto-upgrades have been disabled temporarily."
I’d like to add something like that to StatusGator but I’m unsure if there’s a market.
> "Downtime" means, for a domain, if there is more than a five percent user error rate. Downtime is measured based on server side error rate.
https://status.cloud.google.com/incident/zall/20007
Edit: New incident number https://status.cloud.google.com/incident/zall/20008
Surely no one can send me emails now that gmail is down as most of world is relying on single point of failure, but this is another story.
Moral of this story is - always own your mission critical infrastructure.
Sure, if you have the time and money to own it properly. How far do you need to go to say you own it? Multi-regional servers located on properties that you own?
Especially because Teams status notifications seem funky today too
I'm in Sydney, Australia.
No images either!
Haven't tested the Gmail web interface. I don't use it.
Edit: I seem to be able to send from some accounts but not others. It looks like emails without attachments are fine, otherwise the drive error messes them up.
It's nearly the end of the workday on the east coast of Australia. Very insensitive of you. Sincerely, Western Australia
The above is a joke just in case that wasn't clear.
At least it's not daylight savings yet so the QLDers can't be annoyed at me too!
Earnest question, don't mean to sound derisive.
script connects to drive and parses list of files within
script copies files into redshift and moves original file to different directory or deletes it
alas what can be done? this is what the company gets for building like this ^.^