NanoVMs CEO Ian Eyberg introduces unikernels, a niche deployment method for microservices that take VMs granular.

It seems like the DevOps world is consistently refactoring it’s existence yet at the same time smashing into other domains of the business. Recently, security has been one of those domains creating the new discipline of DevSecOps — that is, applying DevOps principles towards security. This has created an opportunity for an obscure infrastructure paradigm to start to take hold.

Unikernels are an emerging infrastructure deployment pattern that is going in every direction very fast. What is a unikernel though? Simply put, unikernels are the equivalent of deploying your application as it’s own operating system. Since developers predominately deploy their applications on top of Linux virtual machines, unikernels are a way to package your application as it’s own virtual machine in lieu of sitting on top of something like Linux. Is it like a container? Well, there are a few definitions but the concept is relatively old.

The History of The Unikernel

As far back as the mid–90s, noted operating system theorists were already predicting the rise of unikernels. Andrew Tanenbaum even declared Linux deprecated the day it came out, saying it was outdated.

So why didn’t unikernels take off then? Well, back then we were still scaling our servers vertically on real metal. No one even knew what virtualization was, although a form of containers predates even Linux via Billy Joy’s chroot. Then VMWare burst on the scene and introduced commercialized virtualization everywhere.

A few years after that a small online bookstore from Seattle that you might have heard opened up a service called EC2 that forever changed the way developers pushed software to production — all built on virtualization. This finally enabled the concept of unikernels to make sense. Before it would have been silly to dedicate an entire server to one program and it would’ve been silly to try to run the program in kernel mode on bare metal as it could make the server crash.

Fastforward To Unikernels Today

So why are unikernels starting to take off now? There have been a few reasons. One, the rise and adoption of containerization has conditioned many developers to embrace unikernel-like concepts — isolating a given program to a container and embracing things like immutable infrastructure and microservices.

Second, the never-ending onslaught of data breaches in terms of size, scale, and number coupled with other security concerns like cryptojacking have made technical leaders become wary of container adoption without a strong suite of security software.

Attend any big security conference like RSA or BlackHat and you’ll see dozens of container security companies. They aren’t just trying to capitalize off the craze — CISOs are demanding support due to the incredible insecurity that container-based workloads bring.

Isolation and Specialization with Unikernels. (Repurposed from ZDNet)

Other Reasons For The Unikernel Craze

Security aside, there are other reasons for unikernel adoption. The performance of unikernels, their size, and density considerations also offer real benefits.

Small servers can routinely spin up thousands of unikernels on crappy servers. There are numerous papers from prestigious conferences like HotCloud and OSDI pointing to the impressive boot time of unikernels, clocking in less than 10ms, and their context switching reduction allows developers to take advantage of this.

With newer paradigms such as serverless and edge computing, these benefits are widely received. I was actually at a unikernel conference in Beijing earlier last year, and discovered how Alibaba is exploring utilizing unikernels to build out their serverless platform for their cloud.

Types of Unikernel Implementations

There’s something north of 10 different unikernel implementations out there today. I group them largely into 2 categories: Some are fixated on single languages. I call these the purist unikernels. In a perfect world we’d only use one programming language, but like the Tower of Babel, we as developers speak a crap ton of languages.

So, on the other end, we have what we call POSIX unikernels. This is a little bit of a misnomer because what we’re really saying is that they implement the POSIX standard to some degree — at least more so than the purist unikernel varietals. Though purists don’t desire the extra baggage, a huge benefit is that POSIX unikernels don’t require rewrites. You can literally take a MySQL or tomcat app and run it — no rewrite necessary.

Drawbacks of Unikernels

So what are some of the drawbacks of unikernels? There are a few negatives that are real and a few that are fake. Some detractors mention that debugging sucks. I can affirm from spending way too much time with strace and GDB that this simply is not the case. Debugging can be done with the same tooling that you use to debug your applications today.

One very large real problem though was for the longest time unikernels remained inaccessible to the average developer because they required them to be operating systems experts or low-level C programmers. Simply booting hello world would involve finding the right library to link to for the right gcc version, disabling this cflag, enabling that ldflag or cloning a different source tree that was deprecated 2 years ago. In short, there were many painful rabbit holes to the extent that most people would give up after 5 minutes, throw their computer through the window and go live with some sheep off the grid.

Open Source Tooling To The Rescue

OPS: An open source compilation and orchestration tool to make unikernel implementation easier.

Learnability with unikernels was a real problem. That’s also a problem that the new open source tool OPS attempts to address. It allows anyone to run one command and instantly build and boot a unikernel from any ELF (a native Linux binary) to specifically combat this concern. It’s my opinion that developers should focus on building their applications; not spending forever fighting the server gods just trying to deploy a simple application.

Another big problem, in my opinion, is that unikernels are almost always distributed as virtual machines (VMs). You can deploy them straight to a server or run them on top of Linux, but neither of those options makes much sense for performance and manageability reasons. The ones that do run on top of Linux are typically brought up under Kernel-based Virtual Machine (KVM) so it gets hardware acceleration and it’s orchestrated through qemu. For the uninitiated, Google Cloud and AWS are built on forks of KVM.

However, part of the problem is that if you go to AWS or GCE their minimum disk image size is 1Gb. I’ve compiled unikernels in the kb range, but the average Go unikernels we play with clock-in just slightly larger than the go binary itself (10–15mb). Compounded on that is the fact that creating a custom AMI on AWS or other isn’t super fast either – essentially you can create unikernels on the public cloud but the experience is currently not as great as it is if you are in your own private datacenter or on servers you control. If you have bare metal servers — knock yourself out — you’ll be far ahead of any of the cloud users in terms of running these things.

Cloud Industry Momentum Toward Unikernels

However, we are starting to see this problem recede as Google released gVisor and AWS released firecracker within the past year. Both of these projects, while not unikernels, are very heavy nods towards the big cloud providers looking to refactor their ecosystem to support unikernels. I feel that support will continue to grow as adoption grows. Also some of us heavier cloud users tend to forget that over 95% of computing is still locked away in private data centers and then there is the edge which most pundits see largely eclipsing public cloud deployments.

Though there are impediments toward adoption, nothing in the unikernel ecosystem is insurmountable — it is just work that needs to be done. Take the wide variety of cloud companies, as represented by the CNCF cloud-native ecosystem graphic.

In time, a similarly vibrant ecosystem around unikernels could dwarf this size, namely because of two reasons. One, there is already an established virtualization ecosystem. It’s easy to retrofit unikernels on such platforms. Second, unikernels open up entirely new forms of computing that have been inaccessible to developers today. For instance, researchers from places like NEC and Ericsson are interested in them for NFV capabilities.

Takeaways For API Providers

API developers should take particular note to the unikernel ecosystem as it fully embraces many of the leading concepts found in the API world. Take immutable infrastructure for instance; this requires having proper configuration management in place so you can programmatically build, deploy, and tear down your infrastructure.

Unikernels go a step further because while you may or may not have a mutable filesystem in place (some don’t) they all embrace the single process nature of a unikernel. This means the artifact doesn’t even have the capability of running other programs in the same virtual machine.

Unikernels also embrace microservices. Docker and other container implementations helped popularize this concept, enabling non-Linux experts to spin up a handful of services and link them together with relative ease. Being single process applications, unikernels fully embrace the microservice deployment model and are inherently built to run in such an arrangement.

Whether you are a DevOps professional, a serverless aficionado, or API/microservices developer, it’s worth your time to look into unikernels. Spin up a few and get your head around what the coming ecosystem is shaping up to be. We expect a constellation of unikernel projects to emerge throughout the next year.