Rook: Storage Orchestration for Kubernetes | Hacker News Reader

Rook: Storage Orchestration for Kubernetes

github.com

48 pointsjaytaylor13 comments

Rook is a great operator for Kubernetes. We've been using it in our enterprise Kubernetes distribution for a while now and haven't had any problems with it. Ceph is a complex problem, and Rook manages it flawlessly for us.

It's great that when someone wants to have shared storage or they want to have an object store (we only deploy it as block store), the app developer can include K8s YAML to provision block storage or shared storage.

It might be nice to have a "What does Rook do?" explanation. What does it provide over the helm charts for minio, nfs-server-provisioner, cockroachdb, etc.

Do I use it to provision persistent volumes? Databases? Key value stores? S3 compatible blob APIs? Those things all have "persistent storage" in common, but what does Rook do on top of them?

I think the scope for Rook is to provide orchestration and management for storage solutions in cloud-native environments. The way it does that is by providing a set of custom controllers and custom resources (CRDs) that you can use to deploy storage solutions that have "active reconciliation". This is also known as the "operator pattern": https://coreos.com/blog/introducing-operators.html

The ongoing management you get from the Rook operators is something you don't get from a Helm chart. The Rook operators are always running, monitoring the environment, and able to take corrective action if anything goes awry.

After you've deployed one of Rook's supported storage systems into your cluster using these operators, you can also use Rook to dynamically provision storage from these systems for your apps. For example, if you want a volume for your pod, you could declare a PersistentVolumeClaim (PVC) that will result in a Ceph RBD block device being dynamically provisioned, attached, mounted and connected to your pod by Rook.

Dynamic provisioning of other types of storage resources such as databases and buckets, similar to how it's done for volumes, is something I also think has a ton of value, as it would greatly improve the portability of all apps. I think there's great opportunity for integration between Rook and the Crossplane project (https://crossplane.io/) to enable those types of multicloud scenarios.

You may find these "Rook Intro" slides from the last Kubecon to also be informative: https://docs.google.com/presentation/d/1mXFmMZqTF3JV357ICkIG...

And here's the recording of that Kubecon talk: https://www.youtube.com/watch?v=pwVsFHy2EdE&index=2&list=PLP...

(source: i'm a maintainer on both Rook and Crossplane)

Thank you, I'm aware of the operator pattern. I'm not sure what this does for me beyond the things I've just said. There are already Ceph, NFS, and other persistent volume provisioners that integrate by providing storage classes that can satisfy persistent volume claims.

Implementing CRDs and operators for things other than persistent volumes makes sense, too. But does Rook provide a uniform interface for those to do backups/disaster recovery? Or is it just a collection of heterogeneous storage-y things?

Because it seems like the latter reading the docs, and I think that's okay, but the way the page sells it seems inaccurate:

> Rook turns storage software into self-managing, self-scaling, and self-healing storage services. It does this by automating deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management.

Which resources have all of those things automated?

It looks like Ceph is the only one that has any documentation on disaster recovery at all. For that matter, those docs are solely about getting a cluster in a bad state to restart after quorum is lost. Nothing in the documentation describes how to backup, ship backups, and restore backups.

Note that rook is still in beta. I've had pretty bad luck with it not supporting features that I need. First it was not supporting adding additional OSDs on the same node, though I believe that's now fixed. In the end it was that PVCs can't be mounted by multiple pods at a time, even when set to RWX.

In the latest release (0.9) Ceph in Rook has been declared stable.