Meet volume multi-attach, a great new feature in OpenStack Queens

Volume multi-attach is a feature that enables users to attach and access a single block storage volume to/from multiple servers.
The use cases for this functionality include active-active and hot-standby scenarios, when the data on a volume needs to be accessible by multiple servers in order to recover fast in case of failures or being able to handle increased load on the system. Volume multi-attach requires support from two OpenStack projects, Cinder (block storage) and Nova (compute.)

The support for it added in the Queens release (February 28, 2018) represents years of collaboration, so it’s with great pride that we introduce you to it here.

It’s not cloudy anyway

The question often arises whether or not we should add features to the cloud platform that we don’t consider a good fit. It’s even more common for functionalities that are more challenging to add due to previous architectural decisions. Multi-attach was no different.

While the desire for cloud native applications is industry-wide, it’s quite a luxury to rewrite everything to fit. In order to support this transformation period, we needed to make the platform suitable for workloads that are not fully adapted yet.

In the past couple of release cycles, while working to overcome the challenges that prevented us from implementing the volume multi-attach functionality in Cinder and Nova, we had weekly meetings to decide on the design of new APIs and further improvements to the platform.

So, what do you get?

As a starting point, we added a new attach/detach API to Cinder to make the flows more flexible. The new API introduces the concept of ‘attachment’ which you can create, update, complete or delete in order to attach or detach a volume.

You can use the new attach/detach API to attach a volume multiple times in case the Cinder back end you chose reports support for the functionality. In the Queens release, three drivers support multi-attach: LVM, NetApp/SolidFire and Oracle ZFSSA. You can check the Driver Support Matrix for updates on when additional drivers add support.

You need to create a multi-attach volume type to create a multi-attach-capable volume to be able to use this functionality. You can also re-type a volume to be non-multi-attach-capable or vice-versa, but only when the volume is in ‘available’ state. Using the ‘multiattach’ flag during the volume create operation is deprecated starting from the Queens release.

On the compute side, in order to attach a single volume to multiple VM instances you’ll need to run Nova microversion 2.60 or higher. Similarly, to backends in Cinder you’ll find limitations regarding hypervisors in Nova as well. In the Queens release only Libvirt is supporting volume multi-attach and you’ll need either Libvirt >= 3.10 or QEMU < 2.10. For further information please refer to the Feature Support Matrix.

Currently all the subsequent volume attachments will be attached in read-write (RW) mode including the boot from volume case. You can turn the functionality on or off with two new Cinder policies:

volume:multiattach
volume:multiattach_bootable_volume

You can read more about the functionality and its limitations in the admin guide chapters of Cinder and Nova.

Beyond new APIs and functionality, we’ve also started to cover the testing side not just within the projects, but also Tempest. We’ve added test cases like the basic functionality, booting a server from a multi-attach volume, or resizing servers with a multi-attach volume attached to both.

Check out this overview video with a demo from Matt Riedemann:

What’s next ?

The two teams are looking for feedback and discussing improvements to the functionality. Current improvement ideas to discuss include providing support to specify the attach-mode for the secondary attachments. For the latest information, please take a look into Cinder’s specs list for the current release.

For further improvements in the area of testing, we are considering adding more robust integration testing such as writing and reading to/from a shared volume between two VMs.

If you’d like to get involved or have feedback, you can reach the teams on the OpenStack Developer Mailing List, please use the tags ‘[cinder]’ and ‘[nova]’ in the subject line to ensure it gets the right people’s attention. You can also reach out on IRC, each teams has its own channel: #openstack-cinder and #openstack-nova.

Special thanks to

This long-awaited feature wouldn’t have landed without the hard work of the two teams and especially the following contributors: John Griffith, Matt Riedemann, Steve Noyes, John Garbutt, Walter Boring, Balazs Gibizer, Sylvain Bauza, Melanie Witt, Sean McGinnis and Jay Bryant.

Tags: block storage, Cinder, Queens