News

Boston Superuser Awards Nominee: Ocado Technology

Ocado Technologies is one of the nominees for the Superuser Awards to be presented at the OpenStack Summit Boston, May 8-11.

March 29, 2017
image

It’s time for the community to help determine the winner of the Superuser Award to be presented at the OpenStack Boston Summit. Based on the community voting, the Superuser Editorial Advisory Board will review the nominees and determine the finalists and overall winner.

Now, it’s your turn.

Ocado Technology is among the seven nominees for the Superuser Awards. Review the nomination criteria below, check out the other nominees and rate the nominees before the deadline Tuesday, April 4 at 11:59 p.m. Pacific Time Zone.

Please specify the team and organization for nomination.

Ocado Technology is a division of the Ocado Group that writes the software systems powering Ocado.com, the world’s largest dedicated online grocery retailer. Ocado Technology is also working on the Ocado Smart Platform (OSP), an end-to-end e-commerce solution designed to help other retailers around the world go online. Ocado Technology is organized in several departments grouped by areas of competence. The infrastructure department includes several teams actively developing OpenStack instances as well as multiple teams using the instances, tools and applications developed internally. In total, we have dozens of developers and end-users working with OpenStack directly and hundreds indirectly.

How has OpenStack transformed Ocado Group’s business?

Ocado Technology operates a number of highly-automated warehouses from which 230,000 grocery orders go out to customers weekly. Since the inception of Ocado Technology, we’ve learned that simply using off-the-shelf third-party solutions wouldn’t suffice to handle our complex requirements and scale. In order to address our very specific, hard-to-solve and real-world problems related to our fast-growing networking infrastructure, we’ve adopted OpenStack to be able to deploy new grocery warehouses in a timely and reproducible fashion. This required that all of the supporting infrastructure had to be fully orchestrated and all the process to build it had to be fully automated. We just couldn’t afford having a big and complex infrastructure manually crafted and adapted to every use case.

How have you participated in or contributed to the OpenStack community?

We have attended and presented at several OpenStack meetups in the UK, provided feedback on mailing lists and are actively reporting bugs. We have been contributing to the OpenStack community since late 2014.

What open source technologies does the organization use in its IT environment?

We are big fans of GNU/Linux and its various distributions, including Ubuntu. Related to OpenStack, we use and contribute to Ceph, a platform for managing the storage needed to handle the huge amounts of data that we produce, from our customer-friendly websites to our highly-automated warehouses and our IoT-ready vans. Our technology stack includes open source tools and technologies such as Apache Tomcat, Kubernetes, Docker, TensorFlow, Django, CoreOS, NGINX and more.

We also open source some of our existing code; for example, we’ve released a modular, themable collection of components for modern browsers called HexagonJS. We’ve also created an open source game called Rapid Router that helps students learn the basics of programming in Python.

What is the scale of your OpenStack deployment?

The Ocado Technology networking team has deployed over 5,000 CPU cores with 27 terabytes of RAM running 1,750 virtual machines (VMs). After gathering all the requirements, we realized OpenStack provided the best option for our infrastructure needs. We started with a small core deployment using Puppet that catered for three roles: Ceph, OpenStack Controller and OpenStack Compute. We have been expanding the deployment to include more features.

What kind of operational challenges have you overcome during your experience with OpenStack?

As we opened our OpenStack tools to our in-house development teams to deploy their applications, we needed to ensure we made them aware of the expected availability of resources. For that purpose, we create availability zones (AZ) and implemented them such that when one of the AZs has issues, it may impact the complete AZ without affecting any of the other AZs. Each AZ had its own set of resources for all three roles and we have run extensive testing where one of the AZs is allowed to fail completely. Due to the real-time considerations of each warehouse, we have a separate and independent complete stack running locally. We’re currently considering creating regions around them to make the complete process of access simpler.

How is this team innovating with OpenStack?

Our performance testing has shown a 50 millisecond latency from instance-to-instance among different OpenStack-based racks. We didn’t require any form of network address translation (NAT) or network breakout, and the instances had the full network bandwidth available to the hosts. When enabling ECMP, we were able to achieve 17 billion of bits per second of traffic between instances in different hosts on a dual attached 10G server. On the flip side, the solution is single tenant, with any instance seeing the whole estate although access and security rules still work to isolate instances. Having a full BGP network also meant we were able to implement anycast at the server layer. This provided us with the ability to have all the services made HA, without any external load balancing solution or pacemaker solution.

How many Certified OpenStack Administrators (COAs) are on your team?

There are currently no COAs on the Ocado Technology team.

Find out more about the Boston Superuser Awards nominees here, and cast your vote by Tuesday, April 4 at 11:59 p.m. Pacific Daylight Time. Voting is limited to one ballot per person.

Tags:

Superuser
Latest posts by Superuser (see all)