Maxim_Kazmin - Fotolia

Feature

OpenStack Manila: File access storage for the open source cloud

OpenStack Manila is the file level access method in development by the open source cloud platform. What is it, how does it work and when will it be ready?

Chris Evans
By
Published: 05 Jun 2015

April 2015 saw the latest release of the OpenStack cloud computing platform, codenamed Kilo. This release introduced Manila, which brings support for shared file systems into OpenStack to complement its existing storage offerings, extending and improving its ability to consume external shared storage resources.

The Manila project introduces the concept of shared file systems into OpenStack. Until now, the two main storage projects were Cinder and Swift. Cinder provides application programming interface (API) support to manage block storage, specifically systems using block-based protocols, such as Fibre Channel and iSCSI.

Block storage provides high-performance access for virtual machines (VMs) and data, but a Cinder volume/LUN is limited to access by only a single Nova guest (virtual machine). This restriction exists because there is no inherent locking or synchronisation process built into block-level protocols. Block devices also have restrictions on capacity, making them difficult to increase and almost impossible to decrease in size.

Swift provides object storage support, making it suitable for storing large binary objects at scale. However, Swift storage isn’t suitable for transactional data or to store VMs, as objects are typically immutable and updated in their entirety. Object stores are also not usually suitable for small objects due to overheads for data protection methods, such as erasure coding, and are relatively inefficient when using simple protection approaches, such as replicas.

Manila bridges the gap between block and object by providing the ability to map external storage systems using file-based NAS (NFS/SMB) protocols to Nova hosts and guests. File shares can be distributed between hosts and guests, as NAS protocol manages locking and data integrity processes required to provide multiple concurrent access to data.

The Manila project was started in 2012 and developed as a fork of the Cinder project, as many of the concepts and API calls were anticipated to share much in common between file shares and LUNs/volumes. 

Since then the project has evolved, and in August 2014 it was classed as “incubated”, effectively achieving final development status before becoming a full core project of the OpenStack platform.

Although Manila is available in the Kilo release, it hasn’t reached full core status, but that should happen later this year.

How Manila works

The main function of Manila is to provide Nova compute instances with access to shared file-based storage. 

The architecture is designed to use external storage resources, such as commercial hardware from the likes of NetApp and EMC, or to use software-based implementations such as Ceph and GlusterFS.

Read more about OpenStack storage

  • OpenStack is a rising star in private cloud infrastructures, but what about OpenStack storage? Computer Weekly takes a closer look at OpenStack Cinder and Swift.
  • OpenStack and its Swift and Cinder object and block storage lead the way, but are not the only options for building open-source cloud platforms.

Manila effectively provides the orchestration components that manage the creation of file shares and the mapping of shares to Nova compute instances and does not sit within the data path. This functionality is implemented as a set of APIs, a command line interface (CLI) and integration into the OpenStack Horizon dashboard.

Manila uses concepts and terms familiar to anyone implementing NAS, such as share (an instance of a file system), share access rule (an ACL) and security service (eg, LDAP or Microsoft Active Directory). 

In addition, share network is used to describe the networking implementation associated with a share, and is used as one way to implement multi-tenancy support. Networking multi-tenancy is implemented using standard features such as VLANs and VXLAN.

Manila provides automated provisioning of file shares to Nova hosts (the servers running virtual machines). At this stage of project development, mapping file shares to Nova guests (virtual machines) is a manual process, although a number of proposals have been made on how the mapping could be automated.

Using Manila

The use of Manila makes it easier for developers to implement systems that scale at the virtual machine layer, but still need access to shared storage resources to deliver resiliency and availability. These features can be delivered using Manila without having to implement complex application-based replication and redundancy processes.

Manila also provides opportunities for hardware suppliers to make their products valid in OpenStack deployments and, as a result, we’ve seen a significant amount of development time provided by NetApp (on Clustered ONTAP), EMC (VNX) and IBM (Spectrum Scale), as well as OpenStack community members, such as Mirantis. In fact, the project lead for Manila is a NetApp employee. Meanwhile, NetApp and Mirantis have provided 29% and 35% of the source code of the project respectively.

Manila isn’t restricted to deployment in traditional storage arrays, and there is currently supplier development activity on the Ceph and GlusterFS open-source storage platforms. This includes support for protocols outside of traditional NAS (NFS/SMB), for example using device drivers built into the KVM hypervisor that use the native Ceph protocol. The open-source project NFS-Ganesha that implements an NFS server in user space can also be used to abstract underlying NFS server hardware, although this does introduce latency and more complexity into the data path.

It is also possible to implement Manila support without an external storage array, using the generic driver provided with Manila. This driver creates a file share using a Nova VM and external block-based storage, with each file share creating a new VM.

Early days for Manila

Manila is available in the OpenStack Kilo distribution, but as discussed earlier it is not fully adopted as a core project. Users can try out features and see how they fit into their environment (assuming there is driver support), although there are still issues to resolve around networking multi-tenancy and provision of shares to Nova guests.

There is also a lot of thought required about how external NAS storage should be integrated into OpenStack deployments. Manila provides only the conduit for provisioning and mapping new shares. It doesn’t provide (at this stage) any integration with backup, or data protection, other than support for snapshots.

This means organisations need to think hard about data integrity and making use of the hardware features (such as replication) they have become used to in their commercial storage arrays, as well as exactly how shared file storage should be used, be it for VMs or data. Manila does, however, provide a system for a gap that badly needed filling.

Read more on Cloud storage