User Stories

From cryptocurrency mining to public GPU cloud: a transformation story

Two million customers. Data centers in over 15 countries. More than a million GPUs for cryptocurrency mining. A look inside Genesis Mining and Genesis Cloud.

August 6, 2018
image

A lot of startups begin life with one idea then pivot to another, but not many have had as interesting a journey as Genesis Mining. Founded late 2013, co-founders Dutch Althoff and Julian Sprung told the story of how they decided to take the success of the crypto and “funnel it over into something that’s more public cloud consumable, or Genesis Cloud,” says Althoff.

Let’s start from the top: Genesis Mining is an Iceland-based company dedicated to cryptocurrency mining, especially cloud crypto mining. That means that private customers sign up, pay with “normal” money, then operate mining hardware at the company’s data centers. Customers reap returns of this mining in crypto, hauling out gems from the mines without having to install a noisy, hot machine in their homes. Genesis Cloud a cloud company from the founders of Genesis Mining.

The first generation miner was, well, a little artisanal, Sprung admits. “From a classical data center point of view, you might find it unprofessional, but this off-the-shelf hardware it allowed us to quickly get it started and open up those first small data centers.” The team quickly realized the potential and was soon building custom-designed mining hardware with their own server nodes designed for that purpose. Today, Genesis Mining has a completely vertically integrated supply chain, manufacturing their gear in China. They get their their own GPUs with custom tweaks and modifications as needed, making them one of the top customers of both AMD and NVIDIA.

They operate on a huge scale. Genesis Mining started out in 2014 with 25,000 customers, Sprung says those numbers “have really exploded” to roughly two million customers in 2018. Genesis mining operates data centers in more than 15 countries around the world, totaling up to more than a million GPUs for mining.

What are these data centers like? Two main ones serve customers now, in Iceland and Sweden. These hot spots were picked because in Sweden electricity is extremely cheap and in Iceland much of the energy is geothermal “so it’s actually nice green energy (and) cooling pretty much comes for free,” Sprung says. Up next are centers in the United States, Kazakhstan and China.

Sprung offered a look inside the Iceland data center, dubbed the Enigma, where 5-6,000 GPUs operate. “The style of deployment and the setup are a bit unconventional from a data center point of view but it’s very functional,” he says says, adding that an extremely high energy density allows it to operate this way. “It has come a long way from the garage with the GPUs blackened to a motherboard,” he notes. The facility does not have storage nodes or management nodes but they, too, are in the works.

Talking about hardware in the Swedish data center
actual hardware because “this is where we get our hands dirty” we have the mining hardware which is a custom-built server made by Genesis mining, dubbed the Archer at Genesis cloud. The archer node has eight GPUs, a dual-core processor, 16 gigabytes of memory and 100-200 gigabytes of storage “depending on how we we roll it out,” Althoff says.

The set-up comes with challenges. “It’s kind of like deploying OpenStack back in 2010-11 when dual-core was still a thing, so we struggle a little bit with how to add these to our OpenStack cluster,” Althoff admits. One thing that is more popular in our mine for how we deploy on an Archer node is through the use of container orchestration, so we need to be very lightweight and lean and then focus very specific workloads on those nodes. For the more general purpose GPU computing node they use Cirrus, essentially an Open Compute Project GPU node.

For more on the GPU workloads running on (Kubernetes on OpenStack, machine learning and more) catch the entire 36-minute talk below from OpenStack Days Budapest or on YouTube.

Photo // CC BY NC

Tags: , , ,

Superuser
Latest posts by Superuser (see all)