Google accidentally deleted a $125B pension fund's account | Hacker News Reader

Google accidentally deleted a $125B pension fund's account

qz.com

86 pointstoss113 comments

[dupe]

Some more discussion https://news.ycombinator.com/item?id=40304666

> “This is an isolated, ‘one-of-a-kind occurrence’ that has never before occurred with any of Google Cloud’s clients globally,” Google Cloud CEO Thomas Kurian and UniSuper CEO Peter Chun said in a joint statement obtained by The Guardian May 8. “This should not have happened. Google Cloud has identified the events that led to this disruption and taken measures to ensure this does not happen again.”

I'm skeptical. I think this is yet another symptom of the rot that has set in at Google - the controls preventing IC employees from unilaterally causing catastrophic damage are getting dismantled in the name of... who knows what. It's likely that any firewall between ads and other services are getting weaker and I wouldn't be surprised if we get some scandal where Google uses Workspace client data for training or ad targeting. We've already seen similar weakening of the walls with the Chrome team come out during the ongoing anti-trust case.

I opened the article assuming it was some form of user error by UniSuper IT but no, it sounds like a completely internal Google problem if the CEO of cloud decided to weigh in and assure everyone that it's a "one-of-a-kind" occurrence.

count of incidents once the layoffs happened went up :)))

It’s almost always a factor of not having enough people to run business processes… as soon as companies try to cut back and overwhelm employees, they find shortcuts to make their workload still manageable.

But let’s not forget a while ago a story was posted about Google being _the_ company with most middle managers doing pretty much nothing.

But individual contributors get fired. And then the shitshow happens.

But it's not a one of a kind thing...

Sure, one of a kind at this scale but I've heard numerous stories of GCP/AWS terminating accounts with no explanation even when asked for one. However because the customer is small, it seems like it just vanishes in the noise and nothing comes of it. It's quite simple, use a cloud provider as a backup but don't trust your primary data with any cloud provider.

4 copies, 2 with completely different cloud providers, with 2 additional copies being far away from any cloud provider each using different storage medium.

I thought this was posted a few days ago, and by my reading it didn't seem like it was GCP's "fault" in that the issue was GCP failed to stop a bogus configuration update not "GCP blindly deleted the data for an account"?

Their comments I’ve seen are vague enough that we’re all trying to read between the lines and infer what they means. Your reading makes sense, and maybe that’s it, but the specifics are important and I’ve seen other plausible speculation too. They really need to be clear about what happened if they don’t want to take a giant reputational hit from this. Nobody ever got fired for choosing AWS.

The part where it's described as a misconfiguration (on the client's part?) but the head of GCP is apologizing makes it sound kind of like one of those "If you owe the bank $100 that's your problem. If you owe the bank $100 million, that's the bank's problem" situations.

Oops, Accidents Happens.

there needs to be some automated backup option to periodically dump your whole G.suite data to some non-Google service... I'd pay for it.

https://support.google.com/accounts/answer/3024190?hl=en

If I understand you correctly.. You don't need to pay, and these archives can be scheduled to be created automatically, you just have to download them and store on a service of your choice.

I bet it was some stupid AI running in background.

Reply on news.ycombinator.com