How the media giant manages four million cores of infrastructure on OpenStack Ironic.


DENVER — At the first Open Infrastructure Summit, James Penick, architecture director, offered a rare glimpse into the inner workings of entertainment giant Verizon Media.

Verizon Media (formerly Oath) is a subsidiary of the telecom giant that most of us probably use every day: TechCrunch, Huffington Post, Yahoo! and Engadget. Keeping these asset-rich sites up and running requires a heavy underpinning of reliable infrastructure.

Penick describes how and why Verizon Media transitioned from an infrastructure built almost entirely on proprietary, custom platforms to one predominantly powered by open infrastructure. His keynote was one of a series that underlined the importance of bare metal, here’s more on the recently announced OpenStack Ironic Bare Metal Program.

The numbers are eye popping: hundreds of thousands of machines with millions of CPU cores on a private cloud that are mainly managed with open-source tech. Penick paints himself as something of an outsider, noting that a boss told him he was the only software architect he’d ever met who didn’t insist on building everything from scratch myself.

“I told him that I chose to use OpenStack because it exists and it has an active and passionate global community who are always working to improve their product. The value of that force multiplication can not be understated.”

Although the team wasn’t starting from nothing, Penick realized they did have to start with bare metal. “We realized that before we grab our tools and build that house, it’s got to stand on something. It needs a foundation,” he says. “Bare metal is the rebar and concrete which form the foundation of your infrastructure. VMs, Containers, and functions are all awesome, but each platform stands on the one beneath.

While it may seem like the buzz word du jour, Penick says that bare metal is “not glamorous, it’s not the latest flashy, cool, new toy, but you’ve got to deal with it.”  So his team invested there, changed the business to make infrastructure-as-a-service bare metal an option, then a default and then “make it unavoidable.” That’s how he says that what was once a heretical idea became common sense.

And what did they build on it? Virtual machines, Kubernetes, containers and much more. Our production workloads run at all levels of this stack and my dream is to one day push as much of our workloads into higher functions as much as possible.

Penick concludes by encouraging active involvement in open source.

“Don’t just consume; participate and give back. The more you put engineering, operations and architectural talent into contribute experience and perspective,” he says.  The more you help shape and guide the direction of the product to better suit your needs! I hope that our experiences moving over four million cores of infrastructure to open technologies like OpenStack Ironic will help give you the confidence to do the same.”

Check out the whole keynote below.

Photo // CC BY NC