CoreOS, the firm behind the namesake Linux build optimised for hosting containerised applications, has unveiled a distributed storage system designed to provide reliable, scalable storage for container clusters managed by the Kubernetes platform.
Available today as an initial release version, Torus is an open source project intended to address the problem of providing persistent storage for applications and services running inside containers, and especially with scale-out deployments of containers.
Torus is designed to solve some common problems for teams running distributed applications, according to Barak Michener, a software engineer on the CoreOS project team.
"While it is possible to connect legacy storage to container infrastructure, the mismatch between these two models convinced us that the new problems of providing storage to container clusters warranted a new solution," he wrote on the CoreOS blog.
Existing distributed storage systems were mostly designed for small clusters of large machines, rather than the cloud-native approach of using large clusters of small machines, Michener explained.
"Worse, commercial distributed storage often involves pricey and even custom hardware and software that is expensive to acquire and difficult to integrate with emerging tools and patterns, and costly to upgrade, license and maintain over time," he said.
Torus has been developed using a building block approach that can scale out easily, and builds on the etcd distributed key value store that CoreOS previously developed as a reliable way to store data across a cluster of machines. Kubernetes also uses etcd, making it a good fit for Torus.
Torus can be easily deployed and managed using Kubernetes, and the initial release of the software includes Kubernetes manifests to configure and run Torus as an application on any Kubernetes cluster.
For simplicity, Torus provides an interface that appears as a traditional file, allowing storage manipulation through traditional basic file operations, according to CoreOS. However, reads and writes are coordinated and checkpointed through etcd's consensus process.
Currently, Torus supports a block-oriented storage model via a Network Block Device, but CoreOS expects that it will be able to add support for other storage models such as object storage in the future.
"Torus includes support for consistent hashing, replication, garbage collection and pool rebalancing through the internal peer-to-peer API. The design includes the ability to support encryption and efficient Reed-Solomon error correction in the near future, providing greater assurance of data validity and confidentiality throughout the system," Michener said.
The move follows the announcement of a storage framework to support containers by EMC last month. Polly, short for polymorphic volume scheduling, is also an open source project, but is designed to integrate with existing enterprise storage platforms such as EMC's ScaleIO, XtremIO, Isilon and VMAX.
Comcast's £29.7bn winning bid more than twice the £13.7bn Rupert Murdoch valued Sky at just eight years ago
A nuclear strike has been considered, but Bruce Willis is nowhere in sight
Spray-on antenna could enable seamless integration of antennas with everyday objects
Parker Solar Probe, TESS and GOLD missions will deliver exciting data, claims NASA