Stretching the elastic cloud: Evolving OpenStack for scale

In 2012, we made a bet that OpenStack would be the right technology to power the Comcast Elastic Cloud. In 2015 we doubled down on that bet, as our OpenStack-powered cloud continued to grow in size and importance to the organization. As we look forward to the rest of 2016, we’re focused on how OpenStack’s vibrant community will meet new demands of scale, multi-tenancy, and operational efficiency.

Over the past year, our Elastic Cloud team has hit big milestones, and we have had and will continue to have greater impact on the OpenStack community than ever before.

Since 2012 we’ve contributed more than 42,000 lines of code to OpenStack, that includes 611 commits, enough to rank 27th worldwide for all releases.
In May 2015, Comcast won the OpenStack Super User Award, which is voted on by members of the global OpenStack community to honor contributions by a member organization. The award recognized our work to deploy OpenStack at scale, as well as our commitment to engaging with the community.
In October 2015, Shilla Saebi, a Comcast elastic cloud engineer, community lead and member of the Women of OpenStack group, became the first woman appointed to serve on the OpenStack User Committee.
By the end of 2016 the majority of our Elastic Cloud infrastructure will be running the latest OpenStack release, with a smooth upgrade path to subsequent releases.
We believe in giving back to the community by contributing code, advancing the real-world needs of OpenStack operators within the community, and making our team members available to help advance community goals. OpenStack has been a powerful transformative technology for our organization and we hope to enrich its capabilities, not just for us, but for all enterprises, large-and-small.

In keeping with that, we’re turning our attention in 2016 to helping OpenStack evolve in two key areas – scalability and operational efficiency. As more large enterprises turn to OpenStack to meet increasingly large and complex cloud challenges, OpenStack needs to take an evolutionary step forward in how it meets those challenges.

The scalability challenge is a good one to have. For us, it means that an ever-growing number of users within Comcast want to make use of our OpenStack-powered cloud for larger and more complex projects. Our internal cloud now supports more than 500 tenants and we are seeing virtual machine demand more than double each year.

Storage has become a key operational focus for the Elastic Cloud. We want to increase the community’s engagement on the large and growing multi-tenant workloads that OpenStack is increasingly being used to manage in the enterprise environment. We need to be able to provision new tenants instantaneously, often with significant resources, and those tenants need to coexist with other large tenants, with their own large storage needs.

We have been optimizing our multi-tier storage solution – local and network-attached, solid state and spinning disk – to support a wide range of workloads with multi-dimensional needs – capacity, throughput, latency, etc. – all without negatively affecting other tenants. Doing so in a cost-effective manner requires close engagement with multiple storage vendors. Hence there arises a need for robust capacity management, usage monitoring and tenant-visible cost management tools.

The need for workloads to span public and private clouds, with multiple technology platforms in both, requires the adoption of an appropriate higher level abstraction, better orchestration tools and improved developer workflows. Many of our internal customers are pursuing container-based approaches within Elastic Cloud, and we are following developments such as Magnum that promise to bring those advantages natively to OpenStack.

Making it easier to upgrade to new versions of key OpenStack components and building in real backwards compatibility will make it easier for users of all sizes not only to use OpenStack but also to adapt it to meet their changing needs. In production environments like Elastic Cloud, it is imperative that operators are able to take advantage of new features with minimal impact to users.

In our time with OpenStack, the development community has consistently risen to new challenges and demands as OpenStack has grown in popularity, user base and scale. Now it is time to rise again, as OpenStack moves into the next phase of its evolution as a true enterprise-scale cloud solution.

At the OpenStack Summit in Austin, Texas next month, we’ll have a lot more to share about how our OpenStack-powered Elastic Cloud is evolving and expanding. Until then, check back here for more about how we’re making OpenStack work for us.

This post was originally published on the Comcast blog. Superuser is always looking for updates from OpenStack users. Please get in touch: [email protected]

Cover Photo // CC BY NC