Network Function Virtualization test results: Management and Orchestration solutions

In the first post of this series, I shared a short introduction to NFV and its importance, as well as a summary about ETSI’s second NFV Plugtests. In the second post I shared the overall results of the tests and some details from the VIM perspective.

In this third and final post, I’ll share a summary of the results related to the Management and Orchestration (MANO) solutions, which as you may already know are the software pieces (VNF Manager and NFV Orchestrator) that define how virtualized network services are built from the resources provided by the infrastructure through the VIM (VMs, containers, networks, etc), their life cycle, and of course, their behavior.

As you can see from the official report and the table below, 10 different MANO commercial solutions were tested, including six proprietary and four open-source based.

It’s important to note that all MANO platforms included a Generic VNF Manager. This means that even though there is room for Specific VNF Managers at the standards, where a VNF brings its own manager (and there were tests for this kind of set-up), in most cases, a generic VNF Manager can cover all life cycle functions on most VNFs.

Another interesting observation is that at the VIM layer, 90 percent + of the vendors presented a single open-source based solution: OpenStack, while at the MANO layer, only 40 percent of the platforms were open-source based.

Still, as opposed to the first Plugtests edition, where open-source projects participated directly, this time there were two commercial (branded) MANO solutions based on open-source software and both of those are based on Open Source MANO (OSM)

OSM is an ETSI-hosted project to develop an open source NFV management and orchestration (MANO) software stack aligned with ETSI NFV. From my point of view, from the open-source MANO solutions available today (where OSM, ONAP and Open Baton stand out), OSM has reached the highest level of maturity and production-readiness across the industry. It also has a lean approach and a large, growing and diverse community that keeps enhancing the project, which is about to launch its fourth release. That’s why Whitestack  selected OSM as the foundation for our WhiteNFV product. We believe that open source brings a brighter future for our industry and we are committed to launching ETSI-standard NFV deployments in the near future.

Back to the tests:  as we did in the previous post with the VIM, let’s explore the main features expected from the MANO layer during the Plugtests:

From those features, the ones that seem to have reached maturity are related to basic life cycle management and Multi-VIM integration (especially with OpenStack, including orchestration of advanced features like EPA, multi-site, etc.), however, performance management, fault management and scaling features seem to be the next set of features that need to reach that level of maturity.

Let’s take a look a the following table from the report:

As you can see, basic VNF lifecycle testing had 300 tests with 99 percent success, followed by basic performance management (where metrics were grabbed from the VIM), with 176 tests executed and 72 percent success. We also see tests with less amount of participation, support and/or interoperability, with features related to performance management (where metrics were obtained directly from the VNFs), KPI-triggered scaling, and fault management.

In our particular case, with WhiteNFV/OSM, we took the chance to enhance our product by increasing our contributions to the open source project at those features required for this Plugtests event. In particular, we are strongly contributing to the existing OSM’s Monitoring module and with new modules for achieving VNF metrics collection and auto-scaling, as well as other important features that should be starting to appear in Release 4 (April 2018). Plugtests gave us the unique opportunity to be side-by-side with engineers from the VNF vendors, so we started developing some of the features onsite during the sessions, taking advantage of the VNF vendor’s feedback. Thanks to that opportunity we were able to successfully test VIM/VNF-based performance management and auto-scaling.

In summary, as basic life cycle features are working consistently across solutions, we can say that at the MANO level, NFV is ready for prime time, however, we should carefully select the right solution depending on the project’s specific demands beyond basic VNF life cycle, especially when related to performance/fault management and scaling, or other more advanced ones like network slicing and service function chaining.

To close this series, I would like to thank the ETSI Plugtests team for organizing such an important event, and of course, the OSM Community and the OpenStack Foundation, for working so hard on these strong open source projects that enable us to prepare a world-class, end-to-end NFV solution.

If  you’re interested in any of the topics in the series, please leave a comment. Thanks for reading!

Cover Photo // CC BY NC