OpenStack operator spotlight: NTT

We’re spotlighting users and operators who are on the front lines deploying OpenStack in their organizations to drive business success. These users are taking risks, contributing back to the community and working to secure the success of their organization in today’s software-defined economy. We want to hear from you, too: get in touch with editorATopenstack.org to share your story.

Here we catch up with Toshikazu Ichikawa, senior manager of cloud services at NTT Communications, and Shintaro Mizuno, senior research engineer at NTT. Toshikazu speaks from an NTT Communications’ perspective and Shintaro complements with a general NTT Group perspective.

How are you using OpenStack? What kinds of applications or workloads are you currently running on OpenStack?

[TI] We’re using OpenStack to provide a public cloud service called Enterprise Cloud 2.0. Because of the nature of public service, even though we’re not fully aware of all of the applications and workloads of our customers, we do know that it includes an non-mission critical enterprise systems in the energy industry, a system to gather and analyze sensor data in the manufacturing industry and a backup site for disaster recovery from customer branches.

[SM] To add to that, in the NTT Group as a whole we have a variety of OpenStack deployments, including NTT Docomo’s NFV platform (vEPC) and their service platform, NTT Resonant’s web portal service (goo.ne.jp), NTT West’s common service platform (network service servers running on OpenStack), OpenStack integration and support service from NTT Data and NTT Technocross, and a number of internal clouds for application development and research in other group companies.

What have been the biggest benefits to your organization as a result of using OpenStack? How are you measuring the impact?

[TI] The ecosystem to support highly scalable and richly featured orchestration software suites is the biggest benefits for us to provide our cloud beside our networking and data center infrastructure. The number of running VM instances exceeded 10,000+ since we launched last year in eight regions all over the world.

[SM] The major reason for using OpenStack in our network service infrastructure is to avoid vendor lock-in. Many of the current network service servers are vertically integrated and tested by vendors and is hard to unbind the hardware and software (e.g. end-of-life of hardware is EOL of software). This causes lock-in and makes it hard for us to provide new software/services time-to-market. Virtualizing servers and having multi-vendor environment would make operation more complex than before, but we believe we would benefit more by having business agility and flexibility.

What future plans can you share with us?

[TI] We are in the process of expanding our services by adding more components such as Heat, Swift and Trove.

[SM] Expanding NFV infrastructure. Wider adoption/on-boarding to network service platform. Containerization of OpenStack control plane is also a plan for some deployments.

What’s a challenge you faced with OpenStack and how did you overcome it?

[TI] The users migrating their traditional IT or its design concept to our cloud sometimes think of the cloud as a virtualization platform. The concept mismatch, such as a requirement of redundancy or low-level hardware abstraction, tends to cause an inquiry the way how to use and how to avoid an issue. It’s important to guide users so that they design their application and system to align the cloud way.

[SM] The transition to upstream-first was a big challenge for us in the early stages of OpenStack deployment back in 2013. We had around 150 in-house patches to not only fix bugs but also modify some of the features to match our requirements. However, that ended up in huge debt and was hard to maintain for frequent upgrade of OpenStack. We changed the whole process and move to upstream-first model. We had to upstream all of our patches and proposed some of the features we had in-house to the community. Some went in and some did not land upstream. The transition took some time, but both dev and ops somehow overcame the mind shift. Now we only have small numbers of internal patches.
Although our common understanding is to upstream everything we develop, not all proposals get accepted by the community, so our current challenge is to work/discuss with the community to get consensus of our use cases. We currently have two non-official projects that we are actively contributing to make it official projects.
On the deployment side, the lack of OpenStack engineers in operation team made it hard to implement OpenStack in the first place. Our team (R&D team) helped them to design and test their environment. They also got some help from external consulting service providers.

How has your team’s OpenStack deployment grown and developed?

[TI] So far, we deployed more than 1,000 hypervisors in 8 regions. We are expanding our deployment day-by-day as the demand goes. And, we are planning to add regions as well.

How is your team adding to the knowledge and experience of the community at large?

[TI] At every OpenStack summit, our team member had presentations to share our knowledge. And, we regularly attend OpenStack Ops meetup as well.

Can you tell us about a community contribution that you’re particularly proud of?

[TI] We’re currently working on the proposal of an Horizon plugin of Heat dashboard including HOT drag-and-drop generator, which we originally developed for our service and are currently working with Heat team to upstream.

[SM] We now have more than 25 OpenStack contributors, including four core reviewers, within our group and have been contributing to the community on a daily basis. We attend every PTG, Summit, Forum, OpsMeetup and contribute to the community in many different ways including contribution to Working Groups such as LCOO, OpsTelco/NFV WG, PWG and the OpsMeetup Team.
NTT Group is also one of the main supporting companies for the Japan OpenStack User Group and contributes to annual OpenStack Days event and regular meetups. We’ve hosted four local Operator’s workshops, held every six months and will be hosting the global OpsMeetup in 2018.

Talk about how you brought OpenStack into your organization. How did you prepare, what did you need to show or prove?

[TI] It’s been a long story for NTT. For sure, a lot of use cases and global success stories help motivate me and my colleagues.

[SM] Testing, Testing, Testing.
We had to answer questions like “How is OpenStack different from VMware vCenter, CloudStack?”, “Is OpenStack mature enough?”, “Would OpenStack really reduce CAPEX/OPEX?”, “We need this feature in OpenStack!”, “Who would give us 24/7 support (in Japanese)?”, “Which SDN solution is the BEST?”, “Would OpenStack benefit our business?”. We had to test every feature, we had to discuss with the vendors, we had to convince management/executives. We literally did everything we could.

What does “open source” mean to you?

[TI] It’s one of the most important factors in choosing a software to keep evolving and to avoid lock-in.

[SM] It has become a big agenda for our executives and will be an important tool for our network infrastructure innovations. Also a mind-changer for many of our network engineers.

Superuser wants to hear more from operators like you, get in touch at editorATopenstack.org

Tags: NFV, NTT