OpenStack continues to strengthen its commitment to interoperability

Following the OpenStack board of directors meeting in Austin on July 28, the DefCore committee met for a couple of days to continue evolving the OpenStack interoperability program tests.

Interoperability has been a major focus and priority for the OpenStack community and Foundation since its inception, and we’ve hit some big milestones this year. At the OpenStack Summit in Vancouver, we rolled out the revised "OpenStack Powered" program for products and services containing OpenStack software.

To participate in the program, public clouds, distributions, appliances, and other products or services must pass interoperability tests. We made a lot of progress with this program in 2015, and, as of April 1, all new “OpenStack Powered” products are tested against an interoperability standard. You can now see which products and services meet these standards in the OpenStack Marketplace. At the Vancouver Summit, 19 products from 16 vendors passed the interoperability standards. Today, that number of products is 23 with more in the pipeline.

If you’re new to OpenStack and Interoperability, here’s some quick background. The work for establishing, updating, and maintaining the interoperability guideline is carried out by the DefCore Committee, a board backed and community-driven group formed in November 2013. The committee defined guideline specifies the components and capabilities that a product must have. Components are defined by OpenStack code that must be present in the product, and capabilities are checked by API tests. In this way, the Foundation can use the OpenStack brand to ensure that the work of developers is preserved, and that operators and users can enjoy the benefits of a consistent API to deploy applications against.

At the latest board meeting, the Board of Directors approved the 2015.07 testing guideline, featuring two major changes:

a reorganization of the capabilities to better reflect what behavior the tests are checking for,
adding Keystone as a required component with tested capabilities.

After the board meeting,the DefCore Committee held its Liberty Sprint at the IBM campus in Austin. It was an incredibly productive two days during which we planned additions to the new standard, worked on solutions to existing procedural and testing issues, and refined how to best apply the trademark program to protect the interests of OpenStack developers and users.

Working with several project technical leaders, the committee decided to add Neutron networking as a required component (scheduled to arrive in 2016), and addressed existing interoperability issues. To encourage community involvement in evaluating the state of current OpenStack deployments, the committee also decided to adopt RefStack as a test result collection tool that operators and users can report interoperability test results to. RefStack provides and API and client for reporting test results to, and a UI for analyzing and comparing results.

The biggest change to come out of the meeting was the addition of Networking as a required component for the Compute program. Under direction from the Board, we are focusing on Neutron as the only approved networking component. We had the opportunity to have a working session with Neutron project team lead (PTL) Kyle Mestery about the current and future APIs, and plan for ways to encourage future development that will work consistently across the variety of possible Neutron configurations. Later, we did the initial scoring on proposed Neutron capabilities. You can take a look at and participate in the preliminary scoring here. Networking capabilities will be advisory in the upcoming 2016.01 standard, and will be required for all Compute and Platform products in the 2016.07 standard.

We also collaborated with the Nova and Glance PTLs, John Garbutt and Nikhil Komawar about how to manage the continued evolution of the APIs, focusing on cross-project dependencies and challenges. John Dickinson and Matthew Treinish, the Swift and Tempest PTLs, joined us for a conversation about how to expand API test coverage to take advantage of in-tree project testing expertise while providing a consistent testing interface through Tempest. Over the next few months, we will work on adding Swift tests as an external plugin to Tempest, merging multiple test suites under one framework.

On the testing side, Catherine Diep, PTL of the RefStack project, gave us a demonstration of the UI her team has developed. RefStack is a community project that runs tests, then collects and analyzes the results. The RefStack and Infra teams are working on a deployment to https://refstack.openstack.org, but you can access the site now at http://refstack.net. It is now the required site to report DefCore test results for trademark approval.

RefStack isn’t just for reporting DefCore results, it’s also a valuable resource for reporting all API test results as a way to identify widely deployed capabilities and compare them between vendors. We strongly encourage all OpenStack cloud operators to run the test suite and provide us with valuable and anonymous feedback on what capabilities are actively deployed right now. Those results will help us determine what APIs are most important for building interoperable applications on top of OpenStack.

During our sprint we continued to refine the administrative aspects of DefCore. We’re settling into a six-month schedule to match the development and summit cycle. In 2015 we introduced four guidelines, but in 2016 and beyond will settle in to a slower 6-month cadence with a well-defined timeline.

In time for the OpenStack Summit Tokyo this October we expect to have a solid 2016.01 draft ready for community review. You can view the current status of the draft here, which will reflect the new components, capabilities, and the status of flagged tests. (A flagged test is one in which a capability is removed from testing because it is not widely supported or has some testing or upstream bug that needs to be fixed.)

For public clouds, we are proposing an addition to the program where continuous testing can automatically extend a trademark license in accordance with the licensing agreement. The goal is to encourage vendors to verify against the latest standard in a rapidly evolving ecosystem, where deployed code changes frequently to add upstream features or address bugs.

I’d like to thank the participants who came to the DefCore meeting, in particular:

Defcore co-chairs Rob Hirschfeld and Egle Sigler for their commitment to this interoperability program.
Catherine Diep for her efforts in test scoring and running the RefStack project.
Mark Voelker for his sharp attention to detail, in particular with preparing the addition of Neutron as a capability and with subtle procedural details.
Van Lindberg for organizing the capabilities and for his insightful analysis.
Also many thanks for Vince Brunssen, Catherine Diep, and Todd Moore for being our generous hosts at the IBM offices.

If you’d like to know more about the “OpenStack Powered” program and the interoperability efforts of the OpenStack Foundation, you can contact me directly at [email protected].

You can learn about the OpenStack Foundation Interoperability program at https://openstack.org/interop, and can participate in the DefCore committee by signing up for our mailing list at http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee, or find us on IRC in the #openstack-defcore room.

Cover Photo by Carsten Ullrich // CC BY NC