Google Cloud SQL for Postgres
cloud.google.com
cloud.google.com
https://feedback.azure.com/forums/217321-sql-database/sugges...
https://feedback.azure.com/forums/217321-sql-database/sugges...
https://feedback.azure.com/forums/170024-additional-services...
PS: managed MySQL is currently the most requested additional service on Azure:
https://feedback.azure.com/forums/170024-additional-services...
.. and managed Postgres + MySQL are currently the third most requested feature in relation to their managed DB offering
https://feedback.azure.com/forums/217321-sql-database/filter...
Azure's USP is basically the deep integration into the MS ecosystem at all points. Azure is relentlessly promoted to every MS technical professional through all of the learning channels (and MS developer outreach is the best), Azure is integrated into the excellent developer experience of .NET and Visual Studio, hosted SQL Server on Azure is really nice (doubly so because it avoid the licensing quagmire), Azure with PowerShell is a fantastic CLI experience, etc. etc. Lots of reasons to love Azure if you are an MS specialist.
If you are outside the MS ecosystem, then you aren't getting much of that. For you, Azure is a less mature AWS with less support by and for your ecosystem. The situation with hosted MySQL on Azure is one example, but it is a two-way street: Microsoft loves it's own stack more, and folks outside the MS ecosystem love Azure less, partly because AWS is already the established default.
I could list papercuts and issues that I have had with Azure itself, but in each case I'm pretty sure or certain that AWS has had equivalent issues earlier in it's life, so it feels a bit unfair to do that.
Disclosure: I work on Google Cloud (but not Cloud SQL).
I don't want to be overly negative though - releasing now at least let's us play with it and see how it stacks up.
As a point of clarification though Beta => GA is about production hardening. Products can't go to GA without demonstrating that they've met their SLOs continuously for several weeks. Products can take longer than that if they feel they should make more tweaks first (and pgsql certainly qualifies), but we want to both ship quickly and get out of the reputation for perpetual Beta.
This might is bit off topic on this thread.
Seems like there is some confusion going on about the Free tier on a post[1]. Can you please comment there on whether the 'Always free' is free only during the first 12 months or forever?
Also, if I have been using Google Cloud since before today (when free trial was $300 over 60 days), am I still eligible for the 'Always free' tier?
From the page linked and the FAQ, it looks like it but many people on the thread I've linked are thinking otherwise.
Can you please clear that up for us? Thank you!
Another quick question. Is there anyway to see during instance creation/usage if it is part of the free tier?
When I create a f1-micro instance, it shows the usual $5/month billing, but doesn't indicate any kind of 'free tier' usage. How can we know if any service we use will be a part of the free tier for sure? I guess I can wait a few days to see if any costs accrue, but I was wondering if there was a better way for everyone.
As soon as it gets out of beta, it will get an SLA and deprecation policy of hopefully 2+ years notice before it gets discontinued.
No timeline, partially because we don't know ourselves. Our main priority for next few weeks will be ensuring everything is nice and stable. Once we are confident in current feature set we will start adding more. You can watch https://cloud.google.com/sql/docs/postgres/release-notes for news.
Source: I work on Cloud Postgres.
As someone who interacts with PostgreSQL on-prem and cloud deployments frequently, I'm curious on the Hacker News community's thoughts on one question.
AWS has a fairly mature RDS Postgres offering. What would motivate you to use Google Cloud Postgres instead?
I've heard of four reasons from users: not on AWS, don't want to be locked into one vendor, cost, and better support. Do any of these reasons resonate with you or are there others?
The other reasons listed are an order of magnitude less important for my cost/benefit calculus (though things like cost and support will definitely matter to orgs bigger than mine).
Plus RDS was the only reasonable option at that price tier (with high availability and multi-AZ). I think competition was sorely needed.
RDS was pretty much becoming a monopoly on this front.
They have a HA postgres offering (not sure about multi-az) and I thought I'd heard good things about them in the past?
The ability to cap costs.
With AWS, by the time you get a billing alarm, you could be thousands of dollars in the hole and there's nothing you can do about it. Avoiding that risk will help me sleep a little better.
I think Google Cloud is the middle ground between Heroku and AWS. It has potential to really gain traction with developers who want to get stuff shipped and not wear DevOps hats full time. What I'd like to see is Google adopting more modern software on their platform (Python 3) and trying to move things out of beta more quickly. You can't sell beta to management easily.
However, man that Awsrunfile or whatever it is called, and the two versions that are incompatible with the different EB versions, and it's painful... the hosted container solution equally so.
GC's docs and tools just seem so much more straight forward (as do Azure's for that matter).
Strongly agree. I think that using an orchestration tool like Ansible or Cloud Formation is basically essential for AWS, because so many of the services have these interdependencies.
> Even getting ElasticBeanstalk running with Docker isn't really easy.
Yeah, we tried it. Not a great experience, and using an AWS proprietary container hosting system when everyone else is standardising on Kubernetes is not remotely attractive. I'm really looking forward to trying App Engine Flexible and GKE. If we can get really good application container hosting from GCP, then that will be a strong pull to migrating whole stacks.
I would love to see your own cloud offerings on Digital Ocean and on Google Cloud. Hopefully, with that said, we'll also see much more reasonable pricing as well!
Yes, we do start at a higher end $990 a month, but we're very much focused on heavier workloads. Typically users come to us at 100 GB or more. If you're well below that we encourage you to stick with single node Postgres either RDS or Heroku Postgres. Once you get close to a $1k a month spend on those then we can start to make sense to consider.
The price to performance ratio is fairly unbeatable.
Yes, you can absolutely get that much storage for that price. Though what your application needs in terms of memory/cores varies quite a bit on the application. We've seen customers that were spending $800 a month on RDS, migrate and get 2-3x performance boost for the same dollar spend.
Don't disagree at all that if you're at a lower volume of data that RDS is a great option for what you get. Depending on your application needs though as you scale and need more data in cache or more cores doing work, though this all depends on your workload. You can have 3 TB of cold storage and that work fine for your application and cost you under $1k a month on Aurora, or you could have needs with 100 GB of data that you need all queries served from cache in milliseconds. It's very unlikely you'll get that for $260 a month on RDS even at as little of 100 GB of data. It just depends on your application workload, for as long as you can you should just keep scaling up because it's much easier.
We very much focus on when you start to encounter that ceiling where scaling up starts to become prohibitive and performance is key not just storage–which can be as early as 100 GB, and is increasingly common the further you get beyond that.
Surely this is only true for certain definitions of 'production workloads'?
Source: Work on Cloud SQL.
https://cloud.google.com/sql/docs/postgres/high-availability
https://cloud.google.com/sql/docs/postgres/replication/
Any timelines on high availability support? This would be pretty helpful
If you're running a CloudSQL instance, you can use a sidecar container to manage proxying the connection to the database: https://github.com/GoogleCloudPlatform/cloudsql-proxy
"The Cloud SQL Proxy allows a user with the appropriate permissions to connect to a Second Generation Cloud SQL database without having to deal with IP whitelisting or SSL certificates manually."
I think what most people mean when they say that is "don't put the DB in Docker", so don't use Docker to host Postgres. Just because your app is hosted by Docker doesn't make it more or less wise to connect to a database. It just depends on if you need a database or not.
I suspect that they intend that use case to be fulfilled by Cloud Spanner: https://cloud.google.com/spanner/
Edit: Nevermind, it does! https://cloud.google.com/sql/docs/postgres/extensions
For the record, uuid-ossp is a steaming pile and there's no reason to use it for anything on a modern Postgres install.
There's no reason to use it and incorporating it into projects just creates headaches down the road.
[1]: https://www.postgresql.org/message-id/flat/52CF07C2.3080101%...
gen_random_uuid()
Only does UUID 4, which should work for many applications.
Don't get me wrong. GCP as a product is really awesome. I've been using GCE and Datastore for a project and they just work. I just can't trust Google enough to bring all my works to their cloud.
And some of these comments are pretty useless. Like this [1] one. What's even the point? If you can't offer anything interesting, not even an anecdotal data point about reliability, why even contribute?
There's every sign that Google is serious about their IaaS. For example, Google is already dogfooding GCP heavily. According to googlers, a bunch of public Google products run on GCP.
Also, it's worth mentioning that GCP support is very good these days.
Sure, Google is no longer non-evil, and they are deservedly notorious for killing products, but GCP is a different category altogether. They deserve being given the benefit of the doubt in this case, especially as IaaS world really benefits from competition.
(I work on this project.)
I can tell you from personal experience that GCP is suitable for these purposes.
Now, granted, we aren't currently running any hugely important TLDs at the moment, but that won't necessarily continue to be the case going forward, plus we aren't the only ones to be using our codebase ... http://www.donuts.domains/donuts-media/press-releases/donuts...
That's exactly the point. Google doesn't put anything mission critical on GCP, which is a huge red flag for anybody putting mission critical applications in a cloud.
Support for their free consumer products is completely different from a paid enterprise offering like GCP. This is true at pretty much every major company that serves both groups.
I work in GCP and can say that we definitely realize this is an issue and are doing our best to address it. We definitely know that trust has been lost and it takes time to regain it, but hopefully you'll give us a chance again sometime.
I am fairly new but it certainly feels as though GCP is a bit different than the other parts of Google. I suspect this is especially true since Diane Greene came. It definitely feels like everyone knows we need to adopt a different culture to be able to sell to enterprise customers.
https://cloudplatform.googleblog.com/2017/03/reimagining-sup...
Before the reoganization I'd send in quota updates and they'd take 1-3 days. The last quota bump I did took 7 minutes.
They had a rocky start as they were apparently ramping up the support organization, but these days I'm hearing nothing but praise.
They're getting very good at transparency. They have numerous mailing lists [1] ("Google Groups", ugh) for things like product release notes and planned changes. They just opened a public issue tracker [2] that covers the entire Google Cloud Platform.
If you're into Kubernetes/GKE, the Google team is also very active and responsive.
At the simplest level when I buy compute time that looks like everybody else's compute time, I don't want to find after the fact that its provisioning is entangled in some weird corporate ideology that is now impacting my end users, left waiting to be discovered by the unsuspecting victim long after they've made plans around GCP.
(This is a true story from 2 months back, same shit used to happen with App Engine all the time)
Here is a list of likely affected territories: https://support.google.com/a/answer/2891389?hl=en
Google restricts access to some of its business services in certain countries or regions, such as Crimea, Cuba, Iran, North Korea, Sudan, and Syria. If you try to sign in to these services from these countries or regions, the following error appears:
You appear to be signing in from a country where G Suite accounts are not supported.
Certain Google services, such as Gmail, might be available in these countries or regions for personal use, but not for business or education use.
GCP is far from the only cloud to suffer from it and many other services impose this too. The Docker Hub does it too for example.
But anytime this comes up on HN there are a slew of comments saying that support is top notch for GCE, so it's worth an evaluation at least.
If you have to raise a ticket it's a bit more variable, but on the whole I've had every issue resolved satisfactorily.
edit: Seriously a down-vote for asking why someone doesn't like one of the major 3 cloud platforms? These threads can be pretty childish sometimes.
Postgres has it, but it is kind of a pain to setup correctly and I love RDS because I don't have to deal with the setup anymore.
Check out barman from 2ndquadrant, we've been running it in production for 2 years now - setup was crazy easy and the one time I had to do a restore (for the same reason, UPDATE without a WHERE) it was no-fuss to get it done.
Of course, don't let me stop you from using RDS - but there's certainly user friendly backup solutions for Postgres when you're not :)
1) Replication
2) High-availability configuration
3) Point-in-time recovery (PITR)
4) Import/export in CSV format
Volume snapshot restoration is atomic and probably faster, but it gets you only the specific point in time where the snapshot was taken.
I guess the worst case in terms of storage usage in both mechanisms are not that different, but PITR should be fully transparent performance-wise.
When I'm doing surgical work on a DB, I always, always want to do something like :
- SELECT statement showing me the relevant rows (and rowcount=N)
- UPDATE/DELETE statement with a corresponding limit equal to N+1
It gives nice, easy peace of mind that you aren't accidentally wiping out your whole table because you messed up the logic on your WHERE clause. Of course if you made a mistake, you probably still killed N records, but often that is much much better than killing the whole table (and maybe also downstream tables that are linked with FKEY cascade delete!)
Sample monthly pricing:
db-f1-micro instance : $7.56 for compute;
10GB HDD : $0.9 for storage;
No network cost if the database instance is talking to a GCE VM in the same region;
Disclaimer: I work on Google Cloud SQL
>db-g1-small
>db-n1-highmem-2
>db-n1-standard-8
>D32 Database Instance (16GB RAM)
>Tier D0, D1, D2, D4, D8, D16, D32
What's up with cloud instance naming schemes? I get this is kind of similar to Amazon, but man, I bet these are really unwieldy in conversations esp. if someone is new to a platform.
There's at least 2 performance dimensions for provisioning an instance: vCPUs and memory. Often also GPUs, local SSDs, local HDDs. And then you have multiple generations of hardware, which aren't 100% comparable to each other.
So coming up with a "fully systematic" naming scheme is not really possible without listing all the parameters, and then you lose the benefit of having a name.
You'll still see instance size names for a while though. I think D0-D32 are for first generation of CloudSQL (which is MySQL only). db-* are for second generation and match GCE instance names.
https://cloud.google.com/sql/docs/postgres/pricing
Perhaps the difficulty of working this out is from a lack of clarity on what those values would be? Or have I missed something bigger?
Whereas AWS aggressively targets the more traditional enterprise stacks (like running a big single-node RDBMS in the cloud), Google seems to give priority to more modern, decentralized tech.
I don't expect this is universal across all products, but one example.
Edit: not a complete end of the world, but it would be really nice (and completely easy/safe) to have the uuid-ossp extension available.
Is Google contributing anything back into Postgres codebase ?
They could have named something different for the product - Cloud SQL ? seriously.
From what I've heard, Google is running a vanilla version of Postgres. They don't offer any features beyond what mainline Postgres provides (unlike AWS Aurora, which runs modified versions versions of MySQL and Postgres).
As a general thought, it's probably be a good idea to wait (say) 6 months until this new Cloud SQL for PG offering has matured + their team has more PG experience.
Places often contribute back to upstream projects as they get more involved in relevant Community, and it sounds like they're just starting out now.
With so many DBaaS tools available, I'd like to know the best options for things like pricing, availability, features, tooling, monitoring, etc...
If my data were at order of 10GBs, I would choose Cloud SQL, RDS, etc. At order of 100GBs, I would try both Cloud SQL, RDS, etc. and Citus, etc. to see which one fits my usecase. At order of terabytes, I would choose Citus or some other distributed database.
(I'm a former Citus employee and Current Googler in a non-Cloud SQL team)
Both RDS and Heroku are aimed more at more modestly sized database in that you don't really scale beyond the vertical capacity of a single node. The prices are relatively comparable with RDS starts at around $20-30 a month depending on the instance type and Heroku Postgres at $50/month. Both services will get you an HA feature that allows for better uptime through automatic node failover.
Overall Heroku Postgres will probably feel a little like the Heroku platform: a little more polished and a little more "managed", but with fewer knobs to tweak, which can be both good and bad depending on your situation.
It's too early to say how Google's offering will shape up, but it'll likely be in the same vein as these first two. Lack of HA probably means that you should limit your production use of it, although it seems that the team intends to implement that eventually based off the service's documentation and other comments here.
Citus and Aurora come into play when you're looking to scale beyond a single node. Citus Cloud starts at $990 a month so you're not likely to come into it without some non-trivial requirements. Aurora is similar idea.
Citus' killer feature over something like Aurora is that instead of going ahead and forking Postgres wholesale, the product runs as an extension, which means that you're likely to get better compatibility going forward with new Postgres features.
Aurora is "compatible" which means that you'll be able to use psql and get access to common functionality, but are likely to see a divergence in support features. A similar situation is Redshift, which deviated around Postgres 8.0.2 [1], and at this point it's safe to say that it will never catch up.
[1] http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and...
Aurora isn't substantially more expensive than RDS Postgres. Caveat is that you can't run it on the really tiny instance sizes right now. On an r3.large with no reserve pricing you're looking at ~$200pm, or ~115 with reserve.
Disclosure: Not a Google Employee, and I don't work on Cloud SQL
Anyone have any comments or experiences with the Google app engine flex environment for ruby?
It doesn't have all the magic of App Engine Standard and deploy times are slower than I'd like (~5 minutes). But I'm okay with that if I can use my own database, any library I want, and I have full portability.
I did a blog post with some of the stuff that I learned: http://www.thagomizer.com/blog/2017/03/09/rails-on-app-engin...
https://cloud.google.com/sql/docs/features#differences-pg
It looks like there aren't many – you can't use SUPERUSER, and they enable extensions, options, and parameters one by one at request.
Don't think that they care (or should care) much about Postgre. One doesn't go to Azure for that.
So probably not, since this is about supporting Postgres on Cloud SQL.
Seriously, who is this for? I have no idea - SSD VPS like this is about $10/mo ...
Still, this doesn't seem competitive with AWS. The same money buys you substantially more RAM: https://aws.amazon.com/rds/pricing/
Edit: sibling comment points out the actual pricing for google is lower.
(1*0.0413+2*0.0070)*730 + 5*0.17 = $41.22/month
1 vCPU 2GB RAM 5GB SSD(For "Local SSD" the IOPS are a function of the number of devices you attach)
https://cloud.google.com/terms/
Disclaimer: I work on GCP, though not on Cloud SQL.
And while this doesn't have an deprecation policy applicable as a Beta feature, most GA Google Cloud features have a one-year deprecation policy, so even if they were retired, no one would have the rug pulled out.)
Bet hey, keep the uninformed meme going if you like.