Why OpenStack is not big enough

Although the OpenStack Foundation has faced criticism for the ever-expanding number of projects in its Big Tent, executive director Jonathan Bryce believes the growth is necessary.

Bryce also talks becoming an integration engine, changing workloads and cloud native apps in a recent episode of OSpod, a podcast about all things OpenStack.
Niki Acosta of Cisco and Jeff Dickey of Redapt fired off a host of interesting questions. We’ve picked out a few interesting bits and edited for clarity. You can catch the whole 55-minute episode here.

What are the criticisms you hear the most and how are you responding to these criticisms?

One from the last year has been around the Big Tent, that’s been an ongoing transition and how the upstream community is governed… a lot of people have been have been kind of confused about what that means and they don’t know exactly what you know like what does that mean for OpenStack what does that mean for what I should use or not use I think those are all and legitimate questions and concerns.
We definitely deserved criticism on how we communicated around it. Ultimately the decision to the change how the upstream projects were governed was the right one, but the process for communicating and explaining it led to a lot of confusion. Two years ago, projects were defined as “integrated,” which is a word with a specific meaning and would lead you to assume something about how those projects work together and interact together that wasn’t fully accurate. So we were in a confusing situation already.

The different technical leaders looked at that problem and try to solve that problem they also saw that you know there were many many other things that were being built in the OpenStack ecosystem that were really useful and valid that weren’t being recognized in some case were starving for more resources because they were they were integrated you know they were developed the OpenStack way by the same developers but companies were unwilling to use them or put resources on it because there’s the label that we had that it seemed like a stamp of approval, so the goal is really to kind of communicate more clearly what the OpenStack development community was developing and then, rather than just having one label, to have a variety of other data that people can look at sea ok you know is this project being used in production how old is it what kind of testing does it have what kind of documentation does it have so, that people who are building products and services who want to use the code directly actually have more information than just integrated.

Some people say OpenStack is too big— the funny thing is that in some ways OpenStack is not big enough.

One of the things that we’ve been doing over the last year on there’s a on OpenStack.org website under the software section there’s a project navigator we launched at the end of 2015 and it surfaces a lot more of that information. It also highlights the core projects around compute, storage and networking that most of the other projects build on integrate with. It definitely doesn’t solve all the problems, it’s kind of step one and we’re continuing to add more information to that.

Some people say OpenStack is too big— the funny thing is that in some ways OpenStack is not big enough. If you look at the problem set that we’re taking on, you need a massive on set of functionality to automate at scale all of the infrastructure in the world’s data centers — so you’re going to need a lot more than just a virtualization manager or an object storage system — which is where we started out six years ago. You need you need a lot more than that to do governance and orchestration and vertical integration for databases, data analytics, etc…

Traditionally, people called OpenStack a cloud platform, is it turning into an integration platform?

What is changing is the answer to the question: “What is cloud?” If you go back to 10 years ago, cloud was really just sort of extremely elastic virtualization, that was cloud and the only thing you could run on cloud was something that could run in a highly elastic cloud virtualized environment. Now, there are incredible workloads being run on top of clouds.

At the OpenStack Summit in Austin, SAP talked about a production example that runs Siemens Mindsphere, which is basically a control system for industrial manufacturing, on top of the SAP cloud platform which runs on top of OpenStack. Ten years ago nobody would have thought about putting that in a cloud environment at all. The scope of cloud has changed dramatically over the last 10 years, so if we want to stay relevant, then we want to continue to meet those those kinds of needs and the scope of what we work on has to grow as well.

Cloud is changing, how are the workloads changing?

What’s been really interesting to me is to see how the workloads that ran on cloud — there’s still the standard ones, basically web services- websites, APIs, the backend, mobile applications- but then you have this split which is bringing over legacy applications, things that traditionally you would run in a really stable, safe never-changing environment you’re trying to bring those forward into a cloud-hosted model on one end and then on the other end you have this cloud native application development movement, where you’re really exploding your application into micro-services that each run on their own. Instead of one application, it’s actually like you have 30 or 40 applications that make up an end-user product and so those are very very distinct extremes…

The problem set that we’re addressing is very broad, this is a general-purpose technology and you look at something like Linux and the bread and butter of Linux when I started working with it in the late 90s was running a single server. That was basically what everybody that I knew who was using Linux was doing and then there were a few of us who were trying to use it to the desktop as well it was really difficult, but you know like most of the use cases for it were running servers. Now, it’s in my phone, it’s in my car, it’s in my set top box on my TV because these are general-purpose technologies. A lot of times they start out with niche use cases and then as that expands go into crazy things that we never imagined.

It seems like everyone’s fighting right now to be the first citizen of cloud native apps. How do you see the container landscape working with OpenStack?

You’re right and I think that it’s a really foolish approach to the idea. There’s so much opportunity out there, I really think that we are just at the beginning of a completely different way for how we manage data and how we build applications… A survey showed that in the first quarter in the U.S. there were more new mobile customers that came online that were cars than cell phones.
It’s not just about our our laptops, it’s not just about our phones, it’s about everything that we do we interact with is becoming connected. That totally changes what you can build and what you work with in that in it from a development standpoint…
If we all just kind of scramble around and we want to be the first citizen — the one who controls this piece or that piece — we delay the true breakthrough that we should be working together to build.

At the OpenStack Summit we did get a few things that the people were kind of like, “what?” We had Gartner speak about bimodal…We also had Alex Polvi from CoreOS and the cloud computing foundation from Google and they showed on a Kubernetes environment that was running the OpenStack services on top of it. We also showed people running Kubernetes on top of OpenStack…

We have to be careful that we don’t hold on too tightly to our toys and miss the bigger opportunity.

Isn’t one of the benefits of OpenStack is that the possibilities both above and below the stack are endless?

That’s one of the great strengths of OpenStack…There are APIs on top that you can build really cool and powerful applications on, but they’re also APIs underneath so that you can tie in storage like Ceph or more commercial storage systems like Netapp and IBM, so it’s pluggable on on both sides…

We’ve gotten past the point where at least the standard OpenStack infrastructure level, the technology, is no longer the issue.

What we have built, ultimately upstream, in OpenStack is a way of producing software in an amazing global collaboration. If you look at where we run the risk as an industry overall of either missing the opportunity or delaying that inflection point where we get to incredible value, it’s because there are a lot of other a lot of other companies who see these technology points as kind of a proprietary opportunity for them and they want to control it, even if they might have software underneath that’s an open source license, they don’t necessarily focus on building the community as a top priority. We’ve seen that for years now, those communities never end up as healthy and as vibrant and diverse as when building a community is explicitly a top priority. That’s part of what worries me and one of the things that I think could get in our way.

Do you think the Foundation will ever have an opinionated build?

Probably not, but one of the things that that we have started doing is working with different users and with different working groups to come up with more and more detailed reference architectures. The use cases are just so varied. However, when we look at things like data from our user survey what we see that there’s some pretty consistent trends.
People who do big data analytics with Sahara project, most of them run Ironic, and it makes sense. Obviously, if you’re going to be doing heavy analytics, then running those on bare metal gives you some performance improvement. But if you were just coming to OpenStack with not a lot of knowledge and not a lot of background, you might not know to go look at Ironic from the get-go.
So we’re working on more documentation around specific use cases, hopefully when we get to Barcelona in October we’ll be able to unveil some cool new documentation. So a specific distribution or a build, probably not, but much more detail around what we’re seeing a common patterns, yes.

There seems to be a lag in the cultural aspect of being able to develop, deploy and adopt these technologies. Do you think that enterprise adoption has lagged because of those cultural elements? What advice would you give to companies that struggle with the cultural aspect of transforming their business?

We’ve gotten past the point where at least the standard OpenStack infrastructure level, the technology, is no longer the issue. It’s almost all cultural and it’s almost all around how do find the right people and how do you adjust years or even decades of historical procedures to this world.

I’ve told this story before, but it’s the perfect example, about a company that wanted to adopt cloud to speed up their development process. It was taking 44 days for them to get a feature out…and then they built a cloud, rolled it out and after that they were able to do it in 42 days! And it’s because they haven’t changed the culture, they haven’t changed the processes, and once they were able to go back and do that and they dropped the deployment time down to hours. In order to get there, they had to go through and change who had a sign-off authority for every single application change and move a lot of that the front end — so you’re validating a deployment methodology versus a specific deployment of that application. Once you get to that point, you can follow that methodology as many times as you want, hundreds of times a week. When I talk with companies, I tell them it’s really easy to get excited about some change and try to adopt it all at once and that almost always ends up failing…

The two things that I think are really important —to have buy-in from leadership for a transition like this, otherwise it’s too easy for people throughout the organization to dismiss it… And Before you come up with with this this grand strategy, try a couple of pilot projects. This is something (you see with) successful OpenStack projects, they almost always had a specific purpose that they started with, then added on afterwards. Because OpenStack can do so many things, it’s really easy to get paralyzed by the possibilities…It’s best to pick a single application, a single development team and meet their needs and learn in a controlled environment where you have a real world use case but you’re basically getting feedback and a really tight feedback loop around one specific scenario. Then add in as you go along…

Do you see public cloud as a threat to OpenStack?

It’s sort of like saying that that Linux is competitive to Facebook. One is a technology and you can use it to build the other thing or you can use it to build alternatives. Earlier this year, there was an article about OpenStack changes focus from private cloud to telcos. First of all, OpenStack isn’t one person or one entity that makes a decision like that, plus it’s used for a lot of different things and the majority of the deployments are private clouds and usually in enterprises but [there are] around 30 or so public clouds that are running OpenStack. As you you look at how public cloud has evolved since Amazon first kicked off the market in 2006, 10 years ago now, it’s gone from being a really simple menu-driven set of options to dozens and dozens of different services. Some of the things that Amazon has added lately include things like email hosting and DNS hosting, things that hosting companies have done for a long time.

We’re seeing diversification in the public cloud market and you’ll end up seeing hyperscale clouds like Amazon Microsoft and Google that are available for broad general purpose applications, the basic bread and butter of the cloud development model. But you’ll also see public clouds focused in a specific region or in a specific vertical, and that’s where we’ve seen OpenStack being used to build those public clouds. There’s a public cloud in Sweden built by City Networks, built specifically for European Union financial regulations. If you are a a financial services company in the European union and you can run on this public cloud and know that you are meeting the requirements that the EU has in place, which include on locality and rules around around citizen data etc…I think we’ll see continued diversification in addition to the standard hyperscale public cloud model.

What is the foundation doing to address the growing need for security?

Whenever you talk about OpenStack security, it’s a multi-layered question…But within OpenStack specifically there are a couple of groups focused on the security of the OpenStack code, ongoing vulnerability management and the basic code security…Those are the two main focus areas – on development and ops. We’re also putting together a security white paper (editor’s note: look for it in mid-July) around that content.

http://www.youtube.com/watch?v=VWXkm-mLU7E

Cover Photo // CC BY NC