User Stories

Yes it blends: Vanilla Forums and private clouds

Private cloud bolsters growth for the online community software company.

November 5, 2018
image

Moving from the public to the private cloud can be a nightmare of competing technologies and business priorities. As online community software provider Vanilla Forums grew its business, they realized that the private cloud — and OpenStack — was where they wanted to be. Luckily, the company was able to make the jump fairly easily, with COO Tim Gunter tasked with leading the transition.

Initially, the Vanilla Forums team set up their online servers using a public cloud. “It was a single server doing the web service and a single server with a database,” Gunter says, “and off they went.”

Gunter joined soon after, and the company grew from there, evolving how it hosts infrastructure. “It became pretty obvious after a few years hosted on the public cloud that we were paying too much for what we were getting,” Gunter adds. They consulted with their provider who recommended switching to a private cloud, still hosted by them, powered by OpenStack.

Vanilla Forums initially came to life in 2005, with a beta version released in 2002 by Mark O’Sullivan to support his online graphic design and developer community. The open source online forum software gained traction with those user communities through its initial release in 2005 and even powered the Mozilla add-on repository comment system for a while. Version two was written by O’Sullivan and Todd Burry in 2009 and the two ended up making a company out of the technology with the help of a Colorado startup incubator. The company now counts Electronic Arts, HootSuite, Patagonia and Adobe as customers.

The Vanilla Forums team adopted the Grizzly release in its cub days, saving Vanilla Forums a ton of money. “Right out [of] the gate we cut our hosting bill nearly in half by switching to a private cloud,” says Gunter. “We continue to grow and it’s much cheaper to add an entire hypervisor than to add a hypervisor’s worth of public cloud VMs, so we’re happy with it. Overall, it’s been much more stable than our public cloud VMs were.”

Like many folks in development, Gunter is self-taught. He’s built a strong relationship with VEXXHOST, who manages Vanilla Forums’ two private clouds, one in Montreal and one in the US. Gunter counts on the personal support he receives from VEXXHOST’s CEO Mohammed Naser, too.

“I don’t really know what I don’t know,” Gunter says. He bounces potential solutions off Naser, who offers his perspective with solutions that other VEXXHOST customers have successfully employed. “It’s more consultative than just a service provider.”

Gunter and his team also noticed that as hardware got cheaper, their price did not. “We started looking around to see if we could move to a different provider, potentially somewhere in Canada. We were looking to pay in Canadian dollars instead of American dollars, saving a bit of money and fostering a closer relationship with our hosting provider,” he said.

Vanilla Forums uses OpenStack to host their software-as-a-service product, a web application written in PHP. It runs on Nginx and MySQL on logical clusters so the company can host customers based on their specific technical, security, and privacy needs. Gunter estimates there are currently about 50 different clusters, comprised  of cache servers, web servers, databases, etc. grouped together serving websites as the main workload. Vanilla Forums also hosts some secondary workloads, namely a real-time asynchronous message queue, an analytics platform and a dynamically-generated icon service along the lines of Gravatar.

As Europe’s General Data Protection Regulation (GDPR) takes effect, EU customers need to work with companies that fit the legislation’s requirements around privacy and data security. “Canada has been designated by the EU as having ‘adequate privacy laws,’ which means that European companies are okay with us hosting their data here,” Gunter says. “The reasons vary, but increasingly people care where their data lives.”

Vanilla Forums started with just three hypervisors five years ago and expanded to a total of 27 now in two different regions. “We were initially only using Nova, so just VMs, but we’re now using Octavia and Barbican as well,” Gunter says. “We’re looking into some other stuff that OpenStack offers. We’re making use of floating IPs. As OpenStack matures and its features become stable, we evaluate them and see whether it makes sense to either add them or use them to replace what we’ve got. Load balancers are a great example of that.”

Gunter hosts most of his company’s SSL on Cloudflare, but still supports communication to his origin servers over SSL. The Cloudflare origin certificate gets hosted in Barbican, which is then used by the Octavia load balancers. This makes for a convenient way to store the certificate without additional servers or adding it to config management systems.

Early on, Vanilla Forums configured their Nginx servers to serve the certificates directly. They’d have to do an SSL handshake and negotiation, which could slow requests down as well as increase CPU load, which reduced the company’s ability to serve concurrent page views. “Putting that on OpenStack allow us do it outside of our actual VMs,” says Gunter. “It’s still being done by our environment, but it’s offloaded from something that we have to look at all the time and care about. It’s offloading tech debt, in a way.”

OpenStack allows the team to offload SSLs and SNI as well. “Using those two together, being able to spin up a load balancer at will and just connect the SSL certs that are already in Barbican to that load balancer saves us just an incalculable amount of time,” Gunter notes. “We previously had to send a ticket to our provider with the SSL cert, paste it into the ticket body and say, ‘Please give us a new IP and please put this cert with it, and let us know when that’s ready.’ It could be the end of the next day before they could get it done.”

What used to be an incredibly slow, manual process, is now taken care of on-site in three minutes with Octavia.

The ultimate benefits of using OpenStack, says Gunter, come down to cost. Even something as simple as having direct access to their environment and build tools lets a much smaller team stay on top of an expanding business. Even with six times its original hosting footprint, the company hasn’t had to grow its ops team.

In addition, staying vendor-agnostic allows Gunter and his team to move vendors when their business demands it. The transition from an earlier vendor to VEXXHOST was easier than he expected. “It took me half an afternoon, and suddenly we were compatible with this other OpenStack cloud,” he says. “I don’t think it would have been that way if we were on someone’s proprietary cloud.”

When the team needs to provision a new cluster, things are much improved. Instead of opening up a ticket with their vendor, they can do it in-house. “I can now do it in 10 minutes or so by myself, and it will be fully ready to roll,” Gunter notes. No vendor tickets, no outside delays.

He’s not a starry-eyed OpenStack evangelist, either, knowing that the technology still has room for improvement.  “It’s not an easy tool or architecture to use,” Gunter admits. “It’s very complicated, it doesn’t work out of the box, you have to do a ton of configuration, and it doesn’t just sit there and work on its own. You have to constantly mind it and care for it. The release frequency of new versions is rapid, so you need to constantly be patching it and deploying the latest version. It can be a full-time job for any environment of any size.”

Since much of OpenStack is still under development, the team has run into bugs and inconsistencies, too, as well as problems with documentation, conflicts between features and modules, and things that are not quite fully finished. At one point, Vanilla Forums’ Octavia implementation could shut down the entire OpenStack environment, thanks to a bug in the module’s health monitoring code. It was fixed relatively quickly but ended up representing two weeks of lost time.

The team has faced other challenges along the way, including an issue with the security rules not getting applied to VMs. To fix it, they’d have to log into Openstack, add a random security group to the VM, then delete it, at which point everything would start to work again. “It was frustrating because you’d build a cluster and then one of the VMs would just not work, and you’d spin your wheels for a day trying to figure it out,” says Gunter. “OpenStack is so complicated that no one [at our provider] could really figure out why that happened.”

Vanilla Forums is open source itself, so they’re no stranger to challenges like these, and would want it no other way. “We’re big contributors to the open source community,” Gunter says. “All of the work that we put into our core product goes straight to open source and is on GitHub for others to use.”

The company uses PHP, Nginx, and MySQL–all open source products. Plus, they contribute their enhancements to modules like the Fluentd log aggregation tool. “We’ve had to build several plugins for that in order to make it useful for us,” Gunter says, “and we’ve made those open source.”

Gunter and his team don’t contribute directly to OpenStack’s upstream efforts, but they do work with VEXXHOST, who does. Gunter’s advice for anyone considering an OpenStack implementation? “I would say do it,” he says. “If you like money, do it.”

The COO also suggests that companies hire an OpenStack expert to manage its implementation. “You’re going to need an OpenStack expert or find a third-party vendor like VEXXHOST to manage it for you,” he says. “It’s not something you can install and forget.”

According to Gunter, OpenStack works better the more services you combine within it. “We’re starting to realize [that fact] now with Barbican and Octavia and Nova all working together. It’s designed for its features to talk to each other and be more than the sum of the parts,” he says.

He describes OpenStack as an attractive, modern environment. “A lot of developers are keen to work on it, especially considering that it’s embracing containers ” says Gunter. “It can potentially have an impact on hiring, if you say, ‘We use OpenStack and we’re an OpenStack shop.’ You’re going to get candidates who you might not have gotten if you were using VMware, [for example].”

The future looks pretty bright for Vanilla Forums. On the technical front, the team is looking to deploy Gnocchi and use it to replace their third-party analytics service. They’re also talking to VEXXHOST about installing Magnum and getting Kubernetes running to work with containers.

Adding new capacity and regions has been made easier by OpenStack and VEXXHOST’s management. “The way that OpenStack works now, [regions] have been actually a lot easier to manage than I had initially thought with the previous provider,” Gunter said. “Deploying a second data center was something that I was dreading, and it’s actually been quite easy to do.”

Ultimately, the ability to grow as a company without incurring much more staff or infrastructure cost keeps Gunter and his company in the OpenStack game. Moving from the public to the private cloud was only the first step, of course, but working with VEXXHOST on the OpenStack platform made the transition and subsequent growth pretty smooth.

VEXXHOST’s CEO Mohammed Naser will be participating in four sessions — from Kubernetes to deployment tools — at the upcoming Berlin Summit. Superuser is always interested in user stories, get in touch: editorATopenstack.org

Tags: , ,

Rob LeFebvre
Latest posts by Rob LeFebvre (see all)