Teleport: Postgres replicator in Go that works without superuser permissions

github.com

5 points·pedrofranceschi··6 comments

There were no solid Postgres replicator that supports schema migrations and real-time data updates when you don't have superuser permissions on the database (read: AWS RDS), so we wrote our own in Go.

If teleport machine itself dies during replication, you'll have an inconsistent replica, wouldn't you?

No, because changes are saved by triggers that run directly on the database. When the teleport instance is back up, those changes will be batched and transmitted.

I think I get it now. Essentially your replica is always in an inconsistent state, you cannot ever guaranty for anything, that happened on one replica, to be present on another. It's mostly for backups, disaster recovery, when it is ok to lose some data, right?

No. Teleport installs triggers on all tables the first time it starts. After the first start, consistency is guaranteed because triggers will save every data mutation in a separate table (that teleport reads to generate batches for the target instance).

Yes, I understand that triggers will save every data mutation, but no matter what you do second replica is not going to see them synchronously. It's always behind and inconsistent, if first one dies - you lose some data.

EDIT: Maybe you should rename it to something like "trigger-based asynchronous replication", so people would get the right idea about the guarantees it provides.

Reply on news.ycombinator.com