News

How Globo.com scores big with OpenStack Swift

Cost savings and an active community act as big players in the move to go open source.

November 18, 2016
image

Neymar da Silva Santos Jr. has been called the present and future of world soccer. Just 24, the Brazil native wears jersey number 11 for FC Barcelona as he racks up international awards for top scorer.

The infrastructure holding up his official website, neymaroficial.com, has to perform as flawlessly as the sure-footed forward, handling up to 6,000 simultaneous network connections and a vast media library serving over 70,000 photos.

Neymar’s website, on Globo.com, runs on OpenStack Swift. The move to Swift marked a major transformation for Globo.com, the largest media portal in Latin America and one of the top 10 most trafficked websites in Brazil. Globo.com, the front door to media colossus Grupo Globo, offers up a public instance serving around 300 million requests a day to worldwide audiences distributed over more than 70 projects in production.

The journey began in 2013, with what dev ops analyst Caio Brentano dos Passos calls a “shift in mindset” that led the Globo.com team to create an open source project for platform-as-a-service (PaaS) called Tsuru. The team designed the PaaS, whose name means “crane” in Japanese, to depend on object storage in its deploy pipeline and looked into Swift. They were sold on the highly available, distributed, consistent object store that would allow them to store lots of data efficiently, safely and cheaply—important factors as the 60-year old media conglomerate on-boards older archive materials to the web and launches internet-native projects.

As a next step, they built an OpenStack Swift cluster with a few servers and Tempauth authentication. “It worked perfectly,” Brentano says, adding that the results enticed a number of internal clients. To keep operations running smoothly with more volume, they soon adopted OpenStack Keystone for authentication. “Since then, our cluster more than doubled its storage capacity and throughput,” Brentano says.

Today, Swift acts as the wind beneath the wings for many major projects at Globo and its partners, including a pioneering broadcast effort serving and storing videos in 4k high resolution called GloboPlay. The platform also warehouses over 14 million objects for millions of users who create uniforms, flags and crests for their teams in fantasy soccer game CartolaFC. Tsuru uses Swift to store Docker images, allowing hundreds of deploys per day.

Brentano says cost is the first, most obvious benefit of adopting open-source software. “We’re gradually reducing the dependency of expensive enterprise storage solutions. In addition, projects that were using AWS S3 can easily migrate to Openstack Swift, with no loss of quality for the clients.” Another benefit was helping their homegrown PaaS become the main platform to develop and deploy new products.

There are still a few areas to smooth over, he concedes, like monitoring. The team currently monitors dozens of metrics, but needs to improve them to have a better understanding about how each project is performing and help clients optimize Swift use. They are also taking their first steps with billing Swift usage, so they’re studying up and testing OpenStack Ceilometer.

Still, their now-nimble adoption is a long way from where they started. “When we started the object storage project at Globo.com, the concepts related to cloud were new to us,” says Brentano. “We also had that cultural challenge you face when you need to convince people to change the way they do things. But as we showed teams how Openstack Swift works they noticed all the benefits and started testing and adopting it.”

The team got another big boost from the “active and helpful” members of the Swift community, which he calls the most valuable asset of OpenStack. “I am pretty sure that their help is much better than the major paid support services available in the market,” says Brentano.

John Dickinson, Swift’s project team lead (PTL) who has been involved in Swift development since 2009, says that the Globo team’s results underline the project’s versatility.

“One of the highlights of working in OpenStack is seeing development and deployment in every corner of the planet—from NTT in Asia, OVH in Europe, SoftLayer in North America and Globo in South America,” Dickinson says. “I love seeing the passion the Globo team has in providing world-class infrastructure and it’s a pleasure to work with them to make Swift the best object storage system in the world.”

Object storage is one game where everyone comes out a winner.

This article first appeared in the print edition of Superuser magazine, distributed at the Barcelona Summit. If you’d like to contribute to the next one, get in touch: [email protected]

Cover photo by:
(T)imothep // CC BY

Tags: ,

Nicole Martinelli
Latest posts by Nicole Martinelli (see all)