A birds-eye view of Swift 2.20.0

OpenStack Swift is a durable, scalable, and highly available object storage system. It’s designed for storing unstructured data and is a perfect companion to scalable compute infrastructure, whether bare metal, VMs, or containers.

I’m happy to announce that Swift 2.20.0 is now available. This release includes many improvements, but the bulk of the updates are in three key areas: S3 compatibility, encryption, and performance/optimization.

Improved S3 compatibility

Swift incorporated S3 compatibility in the first half of 2018. Since then, we’ve been working on quite a few improvements. One important change we’ve made is to update the way ETag response headers look to better match what S3 clients expect. Specifically, when downloading multipart objects, S3 includes a literal “-” in the ETag, and clients use this information to determine how or if the data is validated after download. Swift’s S3 compatibility layer now matches this functionality, enabling more S3 clients to seamlessly work against Swift.

I’m also happy about an improvement we’ve made to AWS v4 signature validation. Previously, Swift would need to send the signed request to Keystone in order to authorize the account, but now Swift can simply request the signing key from Keystone and validate the request locally. This change allows Swift’s S3 compatibility layer to support many more concurrent connections and requests per second.

We’ve also added some limited support for S3 versioning and updated some default config values to more closely match S3’s behavior.

Encryption updates

Swift has supported at-rest encryption since mid-2016. This feature is designed to protect user data stored on drives to lessen the risk of data leaks if a drive were to leave the storage cluster.

Encryption in Swift uses what we call a “keymaster” to manage access to encryption keys. The keymaster is the piece of code that knows how to fetch the correct encryption keys and where to fetch them from. Swift supports a basic keymaster that stores data in a config file, a keymaster that talks to the OpenStack Barbican service, and a keymaster that talks directly to external key management systems with the KMIP protocol.

In this release, Swift now allows operators to use more than one keymaster at a time. This enables migrations from one key provider to another.

Performance optimization

On the performance side, this release of Swift includes improvements to the erasure code synchronization process. We’ve also added some tuning parameters to several other background processes so that they do not consume excessive CPU cycles in the event they are not IO bound.

Get involved!

I’ve only touched on the highlights from this release. The full changelog is available at https://github.com/openstack/swift/blob/master/CHANGELOG.

This release of Swift is the work of more than 30 developers, including 10 new contributors.

As always, you can upgrade to this version of Swift from any older version with no client downtime. I encourage everyone to upgrade to Swift 2.20.0. There’s plenty to keep us busy as we work on our next release, so if you’d like to join us, please stop by #openstack-swift on freenode IRC.

About the author

John Dickinson has been a project team lead (PTL) for Swift, OpenStack’s object storage service, pretty much since it took off in 2011. At the time he was working at Rackspace, since 2012 he’s been director of technology at San Francisco-based startup SwiftStack. You can find him on Twitter at @notmyname.

Cover image Lefteris Stavrakas – Βουνοσταχτάρα Alpine Swift Tachymarptis melba, CC BY-SA 2.0, Link

Tags: object storage, open source community, OpenStack, Swift