Catching up with the Original Superusers

The OpenStack Community has over 16,000 members across 138 countries. That’s a lot of people supporting the growth and development of the OpenStack cloud platform. Does it take a certain kind of person, or a certain kind of company to become a change agent within one’s own organization and industry, while also contributing to OpenStack’s growth and success?

The Pioneers

If you talk to them, you’ll find they don’t really call themselves superusers, but they have invested a lot of time and energy into figuring out how OpenStack can support their needs, and they have some pretty substantial requirements for their infrastructures.

Take Bloomberg, which on any given day deals with 2 million instant messages and 200 million email messages. It runs one of the largest private networks in the world – 20,000 routers across its WAN) and according to Pravir Chandra, head of security architecture at Bloomberg, has the largest server side JavaScript deployment in the world (22 million lines of JavaScript code in production). Bloomberg also processes 45-50 billion ticks per day of financial data.

Or Cisco, which includes WebEx, Jabber and Cisco Social, is one of the largest business SaaS vendors, supporting millions of meetings for 10s of millions of users, totaling billions of minutes each month. The Cisco WebEx services run 7x24x365.

Then, there’s DigitalFilm Tree, a post, production, creative, consulting and software development company which works on shows that can shoot over 100 TB of film a day.

And let’s not forget about CERN, the European Organization for Nuclear Research. Its Large Hadron Collider is 27 km in circumference, 100 meters under the ground, probing distances at 10 -20 meters to understand how the universe works and what it is made of. They were recording 40 million proton collisions per second generating around 1PB/s.

The CERN Worldwide LHC Computing Grid (WLCG) is massive. It is built on a hierarchical model with CERN at Tier 0. Data is then collected and sent to its 12 Tier 1 sites around the world in near real-time. It also has 140+ Tier 2 sites around the world used for creation of simulated data and analysis. In all, CERN mass-storage systems store more than 100 PB of data.

Each one of these organizations made the decision to work with OpenStack for different, but often very similar reasons, and they have all made a conscious effort to actively participate in and support the OpenStack community.

On Being a Superuser

Tim Bell is CERN IT’s Infrastructure Services Manager, but he’s also an elected Director on the OpenStack Board and member of the User Committee. He says that being a superuser means getting involved with the community and helping to improve OpenStack as it evolves.

CERN started looking at OpenStack in 2011. Last year, it went into production and the installation at CERN has grown enormously.

As a superuser, CERN identifies places where OpenStack can be improved and then it works with the community to define the details with others facing similar needs. As a superuser Bell says you help refine the requirements and help testing the solutions once those requirements have been addressed.

Bell has watched the OpenStack community gradually evolve over time from being a group of enthusiasts to becoming a major momentum in the industry. He has seen participation from a wide range of geographies over a wide range of industries, resulting in a very diverse community.

The key, Bell says, is to give users of OpenStack a voice. This is done in a variety of ways: user surveys, mailing lists, blogs, reviewing blueprints and others. Users can channel their ideas and issues to the people within OpenStack that can work on them. The OpenStack user community runs a survey every six months to get a view of where users are with their deployments and how they are using OpenStack in their environments. Bell said these surveys produce hundreds of comments from around the world. The comments are then fed back into the development community as part of the design process.

As for CERN itself, Bell said they saw areas where they could improve OpenStack, some of which they did on their own, others they implemented in collaboration with other companies. This has allowed CERN’s cloud to scale further and be more usable for the community as a whole.

Today, CERN’s OpenStack cloud is in production supporting over 700 users. It has grown to 65,000 cores across two data centers in Geneva and Budapest. Five engineers support users and keep up to date with OpenStack changes.

Bell encourages users to contribute to OpenStack any way they can, to help define requirements and refine them further. By interacting with other operators and consumers of OpenStack clouds, everyone will benefit.

Superusers Take…and Give Back

When he was asked what it meant to him to be called a superuser, Reinhardt Quelle, Operations Architect for Cisco WebEx, said it feels like it oversells what he does. Quelle says he is simply another community member who is trying to make OpenStack solve his business needs.

Community participation, Quelle says, is the most important part of being a superuser, and that includes regularly meeting and talking to community members inside and outside of OpenStack sponsored events. By talking with each other about what they are doing, everyone sees they are all trying to solve real world problems, that OpenStack is not a science project.

Quelle notes a big shift in focus in the community over the last couple of years from a “let’s try to get it to work” to “let’s get it working well,” in addition to developing additional services. This shift has allowed companies like Cisco to build a lot less, relying on OpenStack to provide more advanced services and expand the ecosystem, letting Cisco concentrate on their business.

Cisco’s entry into the OpenStack community was an evolution, starting with Quelle’s group (Cisco WebEx). When they starting looking at OpenStack in the beginning of 2011 there were only a few pockets at Cisco using it (primarily in networking), so there was not a lot of support. As they talked about what was being built in the OpenStack community and what looked right, things started to change.

What attracted Quelle’s team to OpenStack was that it was an open, flexible, loosely coupled system, making it possible for the community to add things as they needed them. As community members push forward things that are important to them in parallel, others are able to take advantage of the improvements when they become important.

Quelle says Cisco WebEx has a large VMWare deployment and Forklift virtualization works great. But what they found as they started developing cloud-native apps that are targeted as infrastructure as a service was that it made sense to have a platform that supported that model. In the last few years the rest of Cisco has began working with OpenStack, and the number of people creating things with and for OpenStack has exploded. Quelle says Cisco is all in on delivering OpenStack, making sure it runs best on Cisco hardware.

Cisco has chosen to take the time to participate in the OpenStack community. As Quelle points out, it’s in their “enlightened best interests to stay connected”, so when they are working on something, they can compare notes, or make sure they aren’t wasting resources working on something someone else is already. Quelle says that everyone is contributing to something that is growing, to something that is more than a sum of its parts.

How does he recommend someone get started with OpenStack? Quelle says if you are in Silicon Valley there’s a meetup happening just about every week. It’s a way to quickly get plugged into the community and connect with peers doing the same thing. He also recommends you start using it as a consumer and experiment.

Cisco went from two data centers using OpenStack to a multi-tenancy, multi-data center implementation providing centralized services. The primary impact of OpenStack and being superuser, says Quelle, has been the velocity of application development, Cisco is building new features and products faster than ever before.

Superusers Are Chefs

Bloomberg has been part of the OpenStack community for more than a year, and according to Pravir Chandra, the software has matured a lot, with a lot more pieces added to the OpenStack ecosystem that have a lot of promise.

Chandra says when they first got involved in the community there were tons of vendors and startups, but not a lot of mature companies using the software. Today, he says there are many actual operators involved. As a superuser, Chandra says you have to commit to the infrastructure and the project, you have to commit to building pieces and giving back to the community.

A superuser must understand the technical value of what’s on the table today in OpenStack as well as what its potential value will be. Being a part of the OpenStack community, Chandra says, means taking time and allowing your own people to take the time to be involved. Contribute feedback on pieces, share challenges and opportunities with other members.

As an operator of OpenStack, Chandra said Bloomberg (and others) can provide insights into the challenges they face in their day-to-day work, from scaling to sizing of their infrastructure to features, making sure the knowledge gets back into the community or contributing them back if there’s interest. You could even sponsor someone to build them for the community as a whole.

For Bloomberg, a lot has changed since the Diablo/Essex timeframe. They started with a small cluster of six hypervisors running the Essex stack and they had a lot of trouble with it at first.

They looked at a lot of examples of how to stand up a robust highly available OpenStack topology. The amount of work it took to figure it all out resulted in a set of OpenStack Chef Recipes which they contributed back to the community. Chandra says Bloomberg wanted people to learn from what Bloomberg accomplished.

Today, Bloomberg has several hundred servers running OpenStack in a data center and they have plans to expand a lot more. As for the future for Bloomberg and OpenStack? Chandra says the sky’s the limit, noting there are many ways they can use it. He says software defined infrastructure offers huge opportunities for agility and quickly trying out new technologies. “you can iterate very fast and come up with new things in a better, more effective way.”

Chandra says developers building on top of OpenStack can focus on the bigger picture of what they are trying to do. For Bloomberg, he said they are trying to figure out how they expand the array of cloud-based by using OpenStack.

From a technical perspective, he says it’s been fun working with smart people who have cool ideas about what they want to deliver and in what direction they want to take it.

Building a Software-Defined Enterprise

For Guillaume Aubuchon from DigitalFilm Tree, OpenStack was revolution. He says there is a lot of innovation in the entertainment industry, but not in cloud and computing and what it can offer creative professionals of TV and film.

Aubuchon says that OpenStack enabled them to develop their own in-house software for reviews and approvals, dailies, and video. Initially, he says, OpenStack was a way to bring hybridization to that software. From there, it grew into post production services, delivering services and compute remotely to clients. For example, it allowed them to scale up for a large feature like HER on a small footprint and scale back down when it was done.

In the entertainment world Aubuchon says no one wants to wait on anything, so being able to adapt and shift their infrastructure in real-time based on client’s needs is a tremendous advantage.

Aubuchon believes the entertainment industry as whole needs to shift towards open standards and that contributing back to OpenStack and embracing the open source model of contributing/encouraging advocacy is a way to help shift the industry in this direction, to “show them this is the future.”

Transparency is key, Aubuchon says, saying they have writers and directors who feel they are putting their hard fought memories, re-envisioned as content, into the cloud and they need to feel safe about doing that. Everyone is more willing to trust and respect the cloud when it’s transparent.

For DigitalFilm Tree, Aubuchon says adopting OpenStack forced everyone to become a Swiss Army knife. He said that’s the great thing about open source software in general – the cross training. Before OpenStack, everyone had their niche, their tasks. Now, everyone is a software developer, coming up with new ideas to make the company more efficient, driving them forward. They are all learning more about what can be done in the lens of cloud infrastructure.

DigitalFilm Tree has always been a company on the cutting edge of technology. When they first installed OpenStack they were like “okay, we aren’t ready to do a multi-million dollar TV show, but we see its potential.” They watched as OpenStack matured from Folsom to Grizzly, when they knew they were ready to deploy software with it, to Havana, when they started to understand how they could do film and TV on it. Aubuchon says that when you look at how the software has evolved over the span of two years, it’s been amazing.

It started in their software department and from there the rest of the team looked at OpenStack as a broad tool for any application. They evolved from a culture of learning what software to buy to help them, to a new culture of trying to figure out what software they could develop in house or what open source software could they customize to suit their needs.

Now, they ask themselves how they can go into a specific TV show and create a custom ecosystem using the OpenStack cloud, making it easy and more efficient for that client. Aubuchon says that software developer mentally has spread to everyone in the company who supports OpenStack.

Being able to contribute back to the OpenStack community and influence its evolution is important to Aubuchon. He says having access to information and documentation provides the ability to self-educate and elicits trust. Even employees view themselves differently, feeling greater control over their destiny (their career isn’t tied to the success of a company or technology).

OpenStack brought many benefits to DigitalFilm Tree, but Aubuchon says that bringing software definition to all aspects of their business was major: the hardware/infrastructure could stay the same, but the software could change. “In a highly volatile, highly competitive business and you’re dealing with six month shows and you have to design a custom workflow for that show – having an infrastructure that can change means we became more profitable.”

For Aubuchon, OpenStack has given him a future that he says he didn’t think was going to exist three-four years ago. Post production on TV and file was disappearing. OpenStack, and the cloud in general, has given it a new life (delivered in a new way).

Just Do It

With over 16,000 members in the OpenStack community, maybe you are wondering how many of them are superusers. The truth is that we all have a part to play. The pioneers we’ve highlighted here have helped blaze the trail, but they aren’t the only ones, and there is an opportunity for many more to get involved. We’re all on the superuser path.

Everyone agrees that the best way to start with OpenStack is to just start. Plain and simple. Download the software or find a good technology partner, attend a meeting, read blogs, sign up for mailing lists, use it, experiment, play, have fun. You don’t have to contribute code back to the project to be a superuser. Just get started, share your experiences with your peers and keep the broader goals of the community in mind whether you are working with vendors or doing it on your own.

The software-defined enterprise is not a dream, it is a reality for some and quickly becoming one for others. And OpenStack is there to help make it happen.