StorageOS v2 puts ‘mini brains’ in Kubernetes storage volumes

StorageOS has announced general availability of version 2.0 of its persistent software-defined storage-in-containers for Kubernetes.

The big change that comes with version 2.0 is the distribution of intelligence to all volumes in a cluster, rather than being controlled centrally.

StorageOS is software-defined storage that is deployed in a Kubernetes cluster. It uses available storage capacity to create a pool of persistent storage volumes that can be made available to containers in Kubernetes as Persistent Volumes (PV). That brings with it self-service functionality as developers can specify the type of storage needed for their containers in Persistent Volume Claims (PVCs) that are matched to PVs.

StorageOS is deployed as a container and so is portable between environments.

Previously, management of volumes was carried out from a central scheduler on the node, but this could become unwieldy in environments where large numbers of containers were managed.

So, StorageOS has given all volumes in a cluster the intelligence to make decisions about failover, placement, etc, so they are not dependent on the scheduler or control plane. This functionality goes by the catchy name “disaggregated clusters” or “mini brains”.

According to StorageOS CEO and founder Alex Chircop, the move is a response to customer feedback about the volume of containers and rates of change involved, and to avoid impacts that flow from those.

Chircop said: “What we had before was a more centralised control plane model and this created challenges for customers when there were lots of containers and clusters. In that type of situation, there’s always something that impacts on something else.

“With a high rate of change in the environment, it’s difficult for any single decision matrix to cope, so we made it so that every volume is responsible for its own actions.”

StorageOS’s approach to persistent container storage sits somewhere between the two main methods of providing it.

On the one hand, it is possible to use mainstream hardware or software-defined storage products and to expose capacity from them as a PV. That is done using the Container Storage Interface (CSI), which numerous storage vendors – more than 60, at the time of writing – have written APIs for Kubernetes.

And on the other hand, there is Rook. Storage OS is closer to this. Rook is another software-defined method of creating persistent storage that runs in a containerised fashion in Kubernetes. Rook and StorageOS provide a means of building the storage control plane into Kubernetes.

Finally, the addition of Delta Sync is aimed to ensure quick recovery capability for StorageOS volumes in busy environments where the impact of interruptions and restarts can multiply rapidly. It does this by only replicating missed data to the node.

Chircop added: “Delta Sync addresses the need to restart storage volumes after interruptions in a way that makes performance predictable – deterministic – and so there aren’t large fluctuations in recovery operations.

Finally, StorageOS has been made secure by default – that is, access controls, certificates, encryption over the wire are on by default.

Chircop said: “This puts a checksum in every block in every volume to guard against data corruption and to determine what data needs to be brought back in the case of an outage. It can reconverge a volume in seconds.

“It’s for standard data protection scenarios but taking into account that with very large amounts of nodes, there is scope for lots of potential failure.”