Hacker News Reader

Comment by jpgvm | Hacker News Reader

When configuring a number of standby nodes > 1 what replication topology is used? Are {all,none,some} replicas synchronous?

Is the postgres super user available?

What is the list of supported extensions?

Can WAL (physical/streaming) replication be configured to a non-managed postgresql instance? I'm assuming logical replication slots should be supported.

Is there in-built streaming backups/point in time restore?

Any details you can share on how failover and general cluster management is performed?

Are version upgrades supported? Assuming that would use pg_upgrade but is there an option for downtime-less upgrades using logical replication?

View on HN

I'll bet it's async replication with streaming. Anything else costs too much in complexity and performance. Of course the cost of such a setup is a small window where data loss is possible -- but if you're ok with this service you're already ok with data loss because the daily backups lose any add+delete operations you do in a day (unless you have triggers that copy the deleted data in a shadow table)

They have point-in-time restore which means they're archiving the WAL logs continuously. You don't lose an entire day's worth of changes.

Some of these are from our support playbook and I'm working on getting a few others.

> When configuring a number of standby nodes > 1 what replication topology is used? Are {all,none,some} replicas synchronous?

Trying to get an answer of this one.

> Is the postgres super user available?

At this time only our administrative users are superusers for the database. All other administrative tasks should be possible from the default "doadmin" user provided when setting up your database cluster.

You can see a list of current users in the database (including ours) with this command from the Postgres CLI: \du

If some administrative task isn't possible with the "doadmin" user just let us know and we can report that to our engineering team for review and potential future change. We can't promise anything immediately, but we can definitely look into changes long-term!

> What is the list of supported extensions?

address_standardizer address_standardizer_data_us btree_gin btree_gist chkpass citext cube dblink dict_int earthdistance fuzzystrmatch hstore intagg intarray isn ltree pg_buffercache pg_partman pg_stat_statements pg_trgm pgcrypto pgrouting pgrowlocks pgstattuple plcoffee plls plperl (PostgreSQL 9.5+) plv8 postgis postgis_sfcgal postgis_tiger_geocoder postgis_topology postgis_legacy (see note below) postgres_fdw repack (PostgreSQL 10+) sslinfo tablefunc timescaledb tsearch2 unaccent uuid-ossp

> Can WAL (physical/streaming) replication be configured to a non-managed postgresql instance? I'm assuming logical replication slots should be supported.

No, not available.

> Is there in-built streaming backups/point in time restore?

Only daily backups are available with our managed database service at this time. We maintain 7 days worth of backups for each database cluster. Backups can be viewed and restored from the Cloud Control Panel by clicking on your cluster and going to the "Backups" sub-tab. This will restore your entire database cluster to that point in time.

While the backup frequency can not be adjusted to more often/weekly/monthly, you can take point-in-time backups by creating a "fork" of your database cluster. The fork can be placed at a specific point-in-time (reference your logs to find out when the transaction happened modifying the data you want to get back), or "now".

> Any details you can share on how failover and general cluster management is performed?

Nodes are monitored and failover happens automatically with minimal downtime if a node becomes unavailable. Some more info [0].

> Are version upgrades supported? Assuming that would use pg_upgrade but is there an option for downtime-less upgrades using logical replication?

I _think_ all updates may require powering off but will try and confirm.

0: https://www.digitalocean.com/docs/databases/resources/high-a...

I recommend adding these points to the website once you know them.

Especially not knowing whether it's synchronous or non-synchornous replication makes it impossible to design systems on it, as that decides whether you can lose some (likely small) amount of data on a failover, or whether the system guarantees nothing is lost.

I also don't understand another key thing:

On the Pricing page you write "Standby nodes with automated failovers". At the same time, the pricing table offers 0, 1 or 2 standby nodes. So what happens if I buy the offer with the 0 standby nodes, and a failure occurs? Is my data gone? Does DO replace the failed node, and how when there are no standbys?

Kamal from DigitalOcean here. With only a primary node and no standby nodes, a failure will cause a new replacement node to be created with its data being a combination of the latest backup and the write-ahead log. This will get you the most recent data possible. Check out this page for more info on auto failover and how it works with the different configurations: https://www.digitalocean.com/docs/databases/resources/high-a...

How is the WAL stored? If there is a problem with the storage, networking, or DC, you will not be able to get the WAL in order to bring up the DB in a new region. Also, the process of replaying WAL since last backup can take a very long time for a high traffic DB, during which time it is going to be completely unavailable.

That's what a standby server is for, so that it's up to date without replaying the WAL. Nothing unique to DigitalOcean about that.

The WAL and all backups are stored offsite and are completely handled on our end.

Re: availability, that’s right if you only have a primary node and no standby ones. Like manigandham said, there won’t be any downtime if you have a standby node.

Thanks, that answers the question on 0 standbys.

Do you also happen to know the answer on synchronous vs asynchronous replication?

No problem. It’s asynchronous

Thanks. Will you offer a synchronous version in the future?

Since asynchronous means that you can lose previously acknowledged writes when the primary node crashes, which forbids many use cases (for example, most things involving money).

And Postgres already offers synchronous replication modes.

No concrete plans for sync replication that I know of. The team is aware of the need for it but there's not much info to share at the moment.

just curious what version(s) of plv8 you're supporting.

This is super helpful. Agreed with other comments about making this available more readily on the DO Managed DB pages.

Having looked through the set of extensions, wal2json (https://github.com/eulerto/wal2json). That makes sense since there's no way at the moment to get access to replication.

However, the use case for wal2json in addition to the access to replication is that it's the best way that I know of to get reliable change notifications from the database. I personally use this to pipe the JSON blobs into message queues to be consumed by applications.

As a short aside, I don't use PostgreSQL's NOTIFY/LISTEN because, as I understand it, you will not receive messages if there are no connections to the DB -- this is a show stopper for me.

AWS RDS allows this and Google's PostgreSQL does not. I've personally abandoned Google's version first partly because it seems to have stalled in terms of upgrades, and AWS for being way too expensive for what you get. Although it doesn't work for me at the moment, I'm hoping the DO option will work for me in this regard in the not so distant future.

Reply on news.ycombinator.com