Ah yes, it was the customer's fault. Stupid customer using method recommended by service provider.
Google is at fault for how they handled it, but the parents comment is still good advice for anyone reading this thread.
I would say the advice here is, wait until the car is basically stopped before you cross the street rather than trusting it will because you think all the signs say they will. I do think this is solid advice for the production systems as well. All the signs can point to yes, but making extra assurances to avoid the bad path is valid for something critical like your production systems (or your life).
you mean like an email literally saying they acknowledge the situation and won't suspend your account?
I'm not sure how successfully you've dealt with support with companies, but I have almost never been able to rely on the promises of a support agent, as they are simply not the ones actually making the decisions (in this case on whether your account gets suspended or not).
Regularly XYZ does not happen, or not exactly in the way you agreed. They cancel your broadband too late, or your internet starts a few weeks too late, etc. You can try to get compensation, or even sue, but some of the damage has already been done at that point. While this okay for temporarily overpaying broadband, this is probably not the case for a company's main production system, where the risk is very high. The company might not even survive to sue.
Most places outsource their support to global teams don't have access. If you have an AM , in many places, they receive the same notice you receive, also they will personally send it and address it to you with a reminder. They also have a lot more sway as the sales side of every company has the power so they can get around all the process issues.
As the other commenter posted above, if you can have an account manager where you have critical services, it will provide you safety at the cost of a higher spend or minimum spending amounts.
Edit: Had about $1mm in GCP spend and couldn't open a ticket to get windows running properly. It took a week to find the AM and a day after he had it fixed.
I’m pointing to this as evidence https://cloud.google.com/customers
I feel like a lot of the problems that people run into with GCP are avoided with two main components.
Number one you should look seriously at the “enterprise” designation for different products and services if you are looking for long term stability and guarantees about the future (details: https://cloud.google.com/apis/docs/resources/enterprise-apis) and secondly you need to buy actual customer support as an add on if you want to avoid the “I had to hit the front page of HN to actually speak to someone” scenario https://cloud.google.com/support
the main problem with azure/aws/gcp is that their parent companies will track everything you and others do, and then close every related thing because they have enough market share to not give a fuck about you.
There are more than enough blog posts of enterprise customers running into random issues with the big thing, buying "big three, promise edition" isn't much of an assurance.
I'd suspect most of those customers have rapid migration plans written up. For a reason.
I can assure you from experience that most large customers do not. For either major cloud, most large customers will have adopted proprietary services like AWS Redshift or GCP's Cloud Bigtable or Bigquery. None of these have anything like the possibility of a "rapid migration".
Not sure what to say in response here…
I mean, it's an error.
Sure, it's infuriating, and in worst-case could financially harm (or cripple) your business. But Google didn't intend for it to happen. That makes no sense from a business perspective.
So, as others here suggest, you have to plan around the fact that mistakes like this can happen, especially when you're small.
In the car analogy it’s akin to looking the driver in the eyes and them signalling to you it’s safe to cross, only to hit the gas and run you over as you do. No amount of looking both ways before crossing would have saved you.
Credit cards are exceptionally finicky, very susceptible to fraud, and payment can often fail. They're great for one off purchases and services that don't matter if they don't go through - but not suitable unless you're going to check that payments are made, which the OP seems not to have done. Perhaps GCP notifications should be better (I've experience the similar scenario with AWS and everything got fixed within a day) - but surely its up to the customer to make sure they pay.
> however we went ahead and made a manual payment covering all the outstanding amount + extra
(the entire last paragraph of OPs text has those and more details)
I suggest not using and then arguing with and against fantasized stuff when we are talking about a concrete problem for which we have a concrete description, no?
Given that OP paid, of what use is your objection about non-paying customers here?
You also omit this important part:
> They ... assured the project will not be suspended.
That's fair enough - somehow I completely misread/understood that part of the post.
But my main point was that you should never be using credit cards for services that you rely on - particularly when billing/payment is automated and hidden from you. Credit card payments and checks fail regularly (10%+ for recurring payment charges) for legitimate (you don't have funds) and less legitimate reasons (a banks AI system thinking the charge looks fraudulent due to something outside of your account etc). Vendors are likely to stop the service as soon as they can get away with so they don't loose money.
100% a mistake on my part of course; I emailed Linode, explained the above, and they said "sure, is 3 weeks time enough to sort it out? I've delayed any action on your account until such-and-such date but let us know if you need more time".
I've been a Linode customer for several years, paying about $100/month. Just a guy running some small things, hardly a big customer.
If Linode can do it, then surely Google can. Yes, you will probably have the occasional non-payment, but is kicking your people off your platform for a simple administrative mistake/mixup that much better? Accepting the occasional non-payment is better than kicking off people at the drop of a hat. I've since paid several more $100/month to Linode and everyone won in this situation: Linode because they kept a customer who will keep paying, and me because I could get my stuff sorted without having to worry about having my services cancelled.
Google's policy is simply short-sighted on every level, and harms Google's business interests too. The reason they can do things like this is because they have money to spare. Every business that's not swimming in cash like Scrooge McDuck treats their customers like this; the "big tech" company are the exceptions.
No, because Linode is using humans to interact with you. Google will not humans in those situation, their mantra is to automate everything so they can scale up with minimal human effort, even when it doesn't make sense to scale up that way.
Fortunately they have customer support, and 5 minutes after emailing them they resumed my services again.
On GCP you're basically SOL
That is your fault, and your customers have every right to blame you. You are responsible for the due diligence on which providers you use.
Blaming a 3rd party might absolve you of some of the blame, but your still not providing the service you agreed to provide.
And before you try to suggest anything else, the condition here was "having customers you can't let down."
You have customers you can't let down. That's the context.