Hacker News Reader

Comment by brianwawok | Hacker News Reader

Is this production ready, or just an early dev snapshot?

We've currently been testing it at Citus, but have not flipped it to be live for our disaster recovery yet.

We're going to start rolling it out for Forks/point-in-time recoveries first, which present less risk to start. Later we'll explore either parallel restores from WAL-E and WAL-G or possibly just flip the switch based on the results.

On restoration there's really no risk to data. Further we page our on call for any issues that happen such as WAL not progressing, or servers not coming online out of restore.

WAL-G is not yet production ready, but it has been used in a staging environment for the past few weeks without any issues. Once fdr adds parallel WAL support, he plans to take it into production.

Cool cool.

is google cloud storage on the roadmap?

I answered this question in a couple of other places, but: unknown because I don't have use for that yet. https://news.ycombinator.com/item?id=15049527

WAL-G looks like it should be able to talk to Minio (minio.io, S3 compatible thing, also written in go) as a backend instead of S3 itself.

Minio has an interesting feature where it can be a "gateway" to other cloud storage. Google Cloud Storage is one of their specific examples:

https://docs.minio.io/docs/minio-gateway-for-gcs

So WAL-G would talk to Minio, and Minio would transparently proxy that to GCS.

Neat. My concern out of the gate is what would be the perf hit.

I assume I am switching from WAL-E to WAL-G for more perf. But WAL-E speaks GCS. If WAL-G needs an extra hop to do so, may lose some of the point of it..

Yeah, no idea personally. Haven't used the gateway functionality in Minio at all.

That being said, the Minio team seem pretty good with writing performance optimised code. Frank Wessels (on Minio team), has been writing articles about Go assembler and other Go optimisation things recently. eg:

• https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-...

• https://blog.minio.io/golang-internals-part-2-nice-benefits-...

So the performance impact might not be such a problem. :)

There was some mention about resumable uploads in the blogpost which sadly each provider handles differently (that is the GCS layer that supports the S3 API does not accept resumable uploads).

Disclosure: I work on Google Cloud (so I'd love to see this tool point at GCS).

Reply on news.ycombinator.com