Hacker News Reader

Comment by sandGorgon | Hacker News Reader

> The primary reason we migrated away from Google SQL for PostgreSQL is because we discovered a bug that was causing data corruption. This was a known bug that is fixed in newer PostgreSQL versions. However, Google SQL for PostgreSQL is several versions behind. The lack of response from the support acknowledging the issue was a big enough red-flag to move on. I am glad we did move on, because it has been 8 months since we have raised the issue, and the version of PostgreSQL has not been updated to-date

We have had similar experiences on various DB related issues. AWS are miles ahead on anything to do around storage and databases.

View on HN

Google cloud support is PATHETIC.

We had a production Kubernetes outage caused by a bug in Google Cloud. We were paying for gold level support.

They deescalated and reescelated our ticket 3 times.

In the end they closed the ticket and sent me a link to "Architecting distributed application" docs. I think they were trying to be condescending. That idiot should have been canned.

Eventually I threw a big fit in the Google Cloud slack channel and thockin ended up looking at the issue and finding the bug.

There's a Google cloud slack channel? Can you link ?

https://gcp-slack.appspot.com

Tim Hockin is my freakin hero.

LOL. Thanks for the shout out. I am sorry you had a bad experience. Please be assured that we are working on it, and that we take this seriously.

If I may be so bold, where should I direct this complaint so that people hear it:

When you delete a CloudSQL instance, it also deletes the back-ups associated with that instance along with it.

There is also no way to mark a CloudSQL instance as "protected" so one bad keypress can lose you your production database and all backups.

This is... cartoonishly dumb. Clownish.

- Cloudsql instance deletion protection should exist

- Those backups should be preserved somewhere when I delete the instance.

agreed - backups have nothing to do with an instance. i should be able to spin up and down postgresql instances, but backups should be stored (and billed) as per cloud storage rates.

What's even more strange is that the PostgreSQL of GCloud is relatively simple. It's just a https://cloud.google.com/compute/docs/disks/#repds replicated disk. Probably inside a K8s Cluster.

I mean setting this up is extremly simple, in a High availability setting on K8s. So I would guess that a company in the size of Google can actually find out how to add a way to run pg_upgrade to upgrade a cluster (inject something into the image that runs the task....). Even on AWS it will run pg_upgrade with a downtime (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...).

Unless you make the ill-advised choice to use AWS Aurora :(

We did that and only then found out that despite documentation suggesting the contrary [1], there is no upgrade path for major versions of AWS Aurora PostgreSQL. You're stuck with whatever you had when you started. (Unless you want to do a pg_backup/pg_restore which could take weeks.)

[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...

In PostgreSQL performance, Google Cloud Platform "beats AWS by a factor of 2 in all [Aiven.io] tests".

  > A program that produces incorrect results twice as fast is infinitely slower.
    — John Osterhout

Not sure if this comment was about the data corruption bug in Cloud SQL's PostgreSQL version, but the benchmarks we ran at https://aiven.io/blog/postgresql-cloud-performance/ used Aiven's PostgreSQL service on top of VMs from the cloud infrastructure providers and weren't affected by Cloud SQL issues.

The same Linux, PostgreSQL, etc versions were used on all platforms. We didn't use RDS, CloudSQL or other managed services from the infrastructure providers, but the PDF version of the presentation does include comparison of our managed PostgreSQL service with RDS PG and Aurora PG if you're interested.

The article doesn't say the GCP results were incorrect. What's your source?

The article said they suffered data corruption. The Osterhout is à propos.

Aiven is a managed service vendor and uses cloud VMs. This is not a test of the AWS/GCP directly managed Postgres.

Reply on news.ycombinator.com