Hacker News Reader

Comment by kafkes | Hacker News Reader

WAL-G is not yet production ready, but it has been used in a staging environment for the past few weeks without any issues. Once fdr adds parallel WAL support, he plans to take it into production.

View on HN

Cool cool.

is google cloud storage on the roadmap?

WAL-G looks like it should be able to talk to Minio (minio.io, S3 compatible thing, also written in go) as a backend instead of S3 itself.

Minio has an interesting feature where it can be a "gateway" to other cloud storage. Google Cloud Storage is one of their specific examples:

https://docs.minio.io/docs/minio-gateway-for-gcs

So WAL-G would talk to Minio, and Minio would transparently proxy that to GCS.

Neat. My concern out of the gate is what would be the perf hit.

I assume I am switching from WAL-E to WAL-G for more perf. But WAL-E speaks GCS. If WAL-G needs an extra hop to do so, may lose some of the point of it..

Yeah, no idea personally. Haven't used the gateway functionality in Minio at all.

That being said, the Minio team seem pretty good with writing performance optimised code. Frank Wessels (on Minio team), has been writing articles about Go assembler and other Go optimisation things recently. eg:

• https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-...

• https://blog.minio.io/golang-internals-part-2-nice-benefits-...

So the performance impact might not be such a problem. :)

There was some mention about resumable uploads in the blogpost which sadly each provider handles differently (that is the GCS layer that supports the S3 API does not accept resumable uploads).

Disclosure: I work on Google Cloud (so I'd love to see this tool point at GCS).

I answered this question in a couple of other places, but: unknown because I don't have use for that yet. https://news.ycombinator.com/item?id=15049527

Reply on news.ycombinator.com