)
It’s time for the community to help determine the winner of the OpenStack Vancouver Summit Superuser Awards, sponsored by Zenko. Based on the community voting, the Superuser Editorial Advisory Board will review the nominees and determine the finalists and overall winner.
Now, it’s your turn.
T-Mobile is one of seven nominees for the Superuser Awards. Review the nomination criteria below, check out the other nominees and rate the nominees before the deadline Tuesday, April 24 at 11:59 p.m. Pacific Time Zone.
Cast your vote here!
Please specify your team and organization for nomination. Who are the team members?
In the true spirit of supporting T-Mobile’s un-carrier efforts along with minimizing time to market and solving industry pain points around data growth, T-Mobile is deploying OpenStack-based virtual evolved packet core (EPC) so that we can continue to support large scale, consumer-facing data connectivity to devices and better prepare for upcoming 5G deployments. There are multiple OpenStack programs within T-Mobile with a mission to virtualize network functions. In this nomination, we’re presenting the largest use-case which is EPC VNF running on OpenStack. Team Leaders: Suliman Albasheir, Senthil Kaliappan, Rahul Pal, Viraj Silva, John Lazarte and Tom Browning.
How has open infrastructure transformed your business?
T-Mobile is looking for exponential footprint growth in EPC network to serve the growing demands. OpenStack is a key component to address this demand and being used for other NFV programs. Working with OpenStack for NFV improved deployment cost and cross functional interaction among different teams. Having OpenStack as a baseline for telecom core systems helps in automation and shortens time-to-market for launching services. The T-Mobile NFV team has provided a tremendous number of improvements to the NFV implementation of EPC on OpenStack improving stability, increasing capacity and improving availability. These improvements were identified via NFV solution design, lab testing, live traffic and VNF rollouts in production network. Scale: approximately 6,000 bare-metal and VMs.
How has the organization participated in or contributed to an open infrastructure community?
At T-Mobile, we did not contribute on code level in the past but this might change in future. We have identified numerous bugs with various vendors/partners during the solution design/validation phase and worked with partners to provide resolutions at the OpenStack level for the same. All these resolutions were identified, reported and resolved through collaborative NFV design with vendor, NFV lab testing, validation, live traffic stress testing and VNF rollouts in production network environment, thereby making deployment of telecom virtual applications more stable.
The T-Mobile NFV team has worked with many vendors and partners to excel the OpenStack integration for telco-grade NFV. Also, T-Mobile organizations have active/regular participation in OpenStack Summits.
What open source technologies does the organization use in its IT environment?
We use Ansible, Puppet, Red Hat Satellite, Zabbix, Kafka, etc. for managing life cycle but this is not an extensive list of tools that we use.
What is the scale of the OpenStack deployment?
Our OpenStack footprint is supporting virtual EPC production traffic, with around 6,000 bare-metal servers and VMs.
What operational challenges have you overcome during your experience with open infrastructure?
IP/NFV issues
- Parity between EPC PNF and OpenStack based VNF.
- Manageability of VNF from FCAPS standpoint, lengthy VNF validation process.
- Layer2 convergence’s impact on RabbitMQ/OpenStack intra process calls.
- Impacts on VNF state machine from: Transport link bundling design’s behavior during Spine/Leaf failover, lack of bi-directional flow detection, NIC bonding setup issues.
- Uneven load balancing on Intel’s Chip’s hashing for SR-IOV stack.
OpenStack issues:
- OpenStack upgrade is a key challenge.
- Issues with OpenStack Fencing.
- Several issues with RabbitMQ that impacted the VNF availability.
- Issues with CEPH that caused VNF restart.
- Ceilometer services causing the slowness in ceph cluster.
- Issues with Galera DB availability.
- Unstable IPMI/ STONITH network.
- Cinder volume service failure.
How is this team innovating with open infrastructure?
NFV use-case:
Currently T-Mobile U.S. is in the phase of deployment of mobility grade data call processing applications as a VNF within its OpenStack environment. EPC VNF is the largest use case. In this setup, IP infrastructure is still treated as PNF and the EPC application is running as a VNF within OpenStack Domain. IP infrastructure is a tag separated domain for various workstreams needed to bring up the VNF as a workload. VNF itself is a fixed workload and there is no elastic scaling in scope for this iteration. Elastic scaling is planned as future use-case.
Future workloads: 5G, IoT, Private APN, Edge computing, MVNO.
How many Certified OpenStack Administrators (COAs) are on your team?
Two people are certified as OpenStack administrators (Mirantis certification).
Voting is limited to one ballot per person and closes Tuesday, April 24 at 11:59 p.m. Pacific Time Zone.
- Exploring the Open Infrastructure Blueprint: Huawei Dual Engine - September 25, 2024
- Open Infrastructure Blueprint: Atmosphere Deep Dive - September 18, 2024
- Datacomm’s Success Story: Launching A New Data Center Seamlessly With FishOS - September 12, 2024
Superuser
Ashlee Ferguson
