)
When it comes to establishing a web presence, many small businesses really just need simple, great-looking web pages. They don’t have the time or the interest to learn how to use complicated site-building tools.
To address this need, DreamHost developed Remixer, an easy-to-use graphic interface to build websites quickly and easily. The application runs on a Kubernetes cluster deployed on top of DreamHost’s OpenStack-based cloud computing resource, DreamCompute.
Thanks to OpenStack, the Kubernetes cluster is deployed with Terraform, without needing extra plugins or weird extensions. Terraform uses OpenStack APIs to create the virtual machines and network topology necessary for Kubernetes, including all security rules, the persistent volumes for etcd and to store application’s logs. The whole cluster is created with a simple configuration script.
Kubernetes itself runs on custom Container Linux (previously known as CoreOS)-based images as a hypercube single-binary, the preferred deployment form of Kubernetes. A python script downloads the basic Container Linux image, configures it for DreamCompute, adds the single hypercube binary, bundles the image, and uploads it in Glance image service.
Remixer runs a custom version of Kubernetes to work around a known issue dealing with OpenStack volumes. DreamHost developers sent a pull request to Kubernetes upstream to get that fixed, and are hopeful their contribution will improve things for all users in similar situations.
The Kubernetes cluster is made of three types of instances: master nodes, workers, and etcd instances. Many of these instances require persistent volumes from Cinder, and that’s the reason for the custom patch.
Without OpenStack and Kubernetes, Remixer would look very different today or would not exist at all. With the microservices distributed as Docker images, the team can focus on delivering features that add value to the customers instead of imaging bare-metal servers to keep them up with demand.
Master instances are used to provide the high-availability of the main Kubernetes components and they host the ingress controllers used to accept and route external traffic using declarative configuration. The resulting cluster can survive a master node loss. The ingress traffic is also managed by the master nodes, but with the use of an ingress controller which is based on Nginx. Modern browsers will request from the next IP address in the pool if one doesn’t answer. As a result, the ingress management is also redundant if all master IPs are added as DNS records to the serviced domain.
Flannel is used to provide network services for the pods across the node border. Every node registers itself in etcd with the help of register_node service and timer. Fleet is used to distribute services in the cluster along with services activated on a particular machine depending on its role.
The etcd cluster doesn’t use the etcd discovery service, as the initial hosts for the cluster are generated via Terraform. Etcd instances use volumes to store the cluster data, so they won’t go away if the instance is lost. To communicate with the etcd cluster every machine has a proxy etcd instance which is connected to the cluster. The etcd cluster is effectively the localhost for other services on the machine.
Remixer’s Kubernetes cluster offers – right from the start – the dashboard, ingress support with automatic TLS certificates management via Let’s Encrypt, log collection and reporting, cluster health/performance metrics in InfluxDB shipped from Heapster, and Grafana as a UI. The size of the cluster is elastic, with worker nodes added or removed with customer’s demand.
On top of this infrastructure, everything runs Dockerized for the Remixer application itself. On a not-so-busy day there are over 25 pods in the cluster, each running a microservice with specific tasks. The Celery-based worker pods take care of more resource-intensive tasks, like publishing the website and building the page previews. Web pods take care of delivering the front-end application, based on Flux React, while the Nginx pods act as reverse proxies for the Python backend, to integrate with DreamHost APIs to create domains, manage billing, etc. Remixer also requires pods for cache (using Redis) and a RabbitMQ pod.
To complete the app, the DreamHost team designed from the beginning a state-of-the-art analytics service that monitors user experience. This analytics dashboard is designed to help product managers understand how the product is used, in order to minimize user interface pain points and gain visibility on front-end errors. Remixer users are at the center of the development cycle, which is based on hard data.
Without OpenStack and Kubernetes, Remixer would look very different today or would not exist at all. With the microservices distributed as Docker images, the team can focus on delivering features that add value to the customers instead of imaging bare-metal servers to keep up with demand.
Cover Photo // CC BY NC
- How to deploy Kubernetes on your laptop - August 3, 2018
- Why blockchains are good for the cloud, too - May 19, 2017
- Why developers need to ask more questions - May 12, 2017
