Introduction
Virtual Machines(VMs) and containers are at the heart of modern computing, powering everything from cloud infrastructure to local development environments. The concepts we now know as VMs and containers have been around since the early days of computing and were shaped by decades of research, hardware constraints and industry needs.
This post is my attempt to provide a brief history or timeline that covers the evolution of VMs and containers, their common origins and how they developed into the modern tools we use today.
Key Concepts and Terminology
VMs and containers were designed to enable multitenancy — running multiple workloads on the same machine by abstracting the underlying host. They achieve this using different approaches:
Kernel: The Operating System’s main component that interacts with the hardware. It manages system resources like CPU, RAM and connected devices. If an Operating System is a car, the kernel is its engine.
Virtual Machines: Emulate physical machines and real hardware using control software called hypervisors.
Containers: Share the host kernel but isolate applications at the process level.
Hypervisor: Software used to manage virtual machines in a host
Virtual Machine Monitor: Another name for a hypervisor, software that allows running multiple virtual operating systems on the same physical hardware.
x86: A popular family of CPU architectures used in personal computers and servers.
Paravirtualization: A virtualization technique where the guest OS is modified to make it aware of the virtualization layer, allowing it to make explicit calls to the Hypervisor for some instructions.
POSIX — Portable Operating System Interface, A set of standards for UNIX-based systems to ensure compatibility between operating systems.
Namespaces — An isolation feature that allow processes to have their view of certain system resources such as filesystems, network interfaces and processes without being affected by other processes.
Early Foundations (1960s-1980s)
In the early computing days, software was written for a particular hardware architecture. When the hardware changed, software had to be re-written and needless to say, this was unsustainable. To address this problem, researchers proposed keeping the kernel simple, stable and managed by a few experts. This would allow the kernel to handle hardware interactions while minimising what programmers needed to know.
During this time, key developments included:
- Improved memory isolation in hardware.
- Early timesharing systems that emphasized isolation
- Rise of Personal Computers led to the decline in VM research
The Decline and Revival of Virtual Machines (1990s-2000s)
Although VMs existed in the 1980s and 1990s, interest waned during this period. The term “virtual machine” was even repurposed by Java and Smalltalk, something that was indicative of how dead the original concept of a VM was in this period. However, the late 1990s saw renewed interest due to research projects that sought to make Operating Systems more portable.
The Disco Project (1997)
A Stanford University project that explored using Virtual Machines as a way to run operating systems on different types of hardware without extensive modifications. The project prioritised portability over security or performance.
VMWare (1999)
The team behind the Disco Project started VMWare a year later and in 1999 released a workstation product and two server products (VMWare GSX Server and VMWare ESX).
The VMWare team faced a challenge virtualising the x86 architecture of the time because x86 processors (from Intel and AMD) weren’t built for virtualization.
Normally, a Virtual Machine Monitor(VMM) or Hypervisor relies on a method called trap-and-execute to control how guest operating systems interact with hardware. VMWare couldn’t trap all the instructions because some could bypass their VMM.
The x86 architecture had a problem:
- some instructions it processed were sensitive but not privileged
- Privileged instructions such as changing memory settings trigger an error (a trap) when run in user mode. When this happens, the VMM catches the error and safely handles it(execute).
- Sensitive instructions have potential system stability and security issues. These instructions don’t always trigger a trap, so they could be run without the VMM noticing, introducing potential security issues.
VMware’s solution:
- Trap-and-Execute where possible. For privileged instructions that triggered traps, VMware would handle them using traditional virtualization techniques.
- Dynamic Binary Translation. For sensitive and tricky instructions that didn’t always trap, VMware rewrote the instructions on the fly before execution. Doing this allowed guest operating systems to run more efficiently, without realising they were virtualized.
- This approach allowed running guest operating systems unmodified and was a breakthrough at the time. Later, Intel and AMD created processor extensions in their x86 CPUs to provide hardware support for virtualization making VMWare’s workarounds unnecessary.
Denali (2002)
A University of Washington project that introduced paravirtualization, a technique that modified a guest OS to work more efficiently with a hypervisor. Since x86 hardware lacked native virtualization support at the time, paravirtualization offered a workaround by modifying CPU instructions. By making the guest OS aware of the virtulization layer,it could communicate more efficiently with the hypervisor. Instead of attempting to execute restricted instructions directly, the modified guest OS would make explicit calls to the hypervisor, allowing it to handle operations that required hardware-level execution.
Xen (2003)
A University of Cambridge project that used paravirtualization techniques with an emphasis on keeping user applications unmodified. Unsafe or privileged calls were replaced with its own hyper calls. Xen introduced features that allowed tracking the resource usage of each guest, something that directly led to the creation of AWS EC2 a few years later.
x86 Hardware Virtualization Extensions (2004/2005)
Due to the rise in the popularity of virtual machines and the complexity associated with running them on x86 hardware, AMD and Intel introduced (independently of each other) extensions to their processors in the mid 2000s enabling hardware support for virtualization. These extensions allowed running VM code directly and trapped any sensitive instructions making paravirtualization or binary translation unnecessary.
AMD-V – the virtualization extension added to AMD processors around 2005. AMD-V debuted with AMD Athlon 64, Opteron and Turion 64 processors. It aimed to eliminate the need for binary translation.
Intel VT-x – the virtualization extension first introduced to Intel processors in 2005 in Pentium 4. It allowed running hypervisors like VMware and KVM efficiently without using complex workarounds like Binary Translation. VT-x allowed guest OSes to run in kernel mode safely and introduced features that intercepted and emulated privileged system calls.
Hyper-V (2008)
In 2008, Microsoft released Hyper-V; their own Hypervisor that took advantage of the x86 virtualization extensions created by Intel and AMD.
Decline of VMs and the Rise of Modern Containers
By the late 2000s, container-based solutions started gaining traction, reducing the reliance on hypervisors like QEMU+KVM, VMWare, Hyper-V and Xen.
The Rise of Modern Containers (1982-2013)
Containers evolved from multiple isolation technologies that were developed around the same time as VMs, including chroot, FreeBSD Jails, resource controls, namespaces and zones.
POSIX Capabilities (1990s)
In the 1990s, the POSIX Group proposed the POSIX.1e standard to add capabilities to the traditional UNIX security permissions. Capabilities allowed users to start processes or perform tasks using a subset of root privileges. The goal of the standard was to split root permissions into units or subsets that could be managed easily and assigned selectively to processes.
The proposal was never accepted and later abandoned. Despite this, the Linux kernel introduced capabilities in version 2.2 in 1999. The Linux implementation wasn’t an exact match to the POSIX.1e proposal but it retained the core principle of splitting root privileges into smaller units or capabilities. Linux is the only major OS to implement in a form similar to POSIX.1e. It expanded them over time to make them more useful to modern security models.
Early Isolation Techniques
Chroot and FreeBSD Jails (1982-2000)
UNIX v7 introduced Chroot, a program that allowed creating virtual filesystems by changing the root directory for running processes. Chroot limits a process to a specific directory, effectively changing the process’ world view and limits the process and its children to that new directory and its subdirectories.
In 2000, FreeBSD 4.0 introduced Jails, enhancements to chroot that extended chroot to isolate not just the file system, but also network interfaces, process visibility and user credentials. Each jail could have its own root user with full permissions inside the jail and no permissions outside the jail.
Linux VServer and Virtuozzo (Early 2000s)
Linux-VServer and Virtuozzo were early container-like virtualization technologies that introduced process isolation and resource control mechanisms. Linux-VServer was a lightweight virtualization project that enabled multiple isolated environments to run on a single Linux Kernel. Virtuozzo, developed by Parallels was a commercial container-based virtualization platform focused on resource sharing in hosting environments. They introduced mechanisms to set limits on resources such as CPU, RAM, Disk, and other resources.
For example, the projects introduced the following isolation mechanisms:
- Memory Isolation – Implemented control systems that prevented containers from accessing each other’s memory.
- Process Isolation – Ensured that a process in one container could not see, interact or interfere with processes from other containers or the host.
- Network Isolation – Enabled isolation of network addresses, allowing processes (or containers) to have their own sets of IP addresses and network interfaces.
This isolation work laid the groundwork for modern containers.
File System Mount Namespaces (2002)
Linux introduced its first namespace, the mount namespace in 2002. Mount namespaces enabled creating isolated file system environments where processes could mount and unmount file systems without affecting the host system or other processes. The ability to isolate filesystems this way enabled processes to have different views of the filesystem, a crucial step toward isolating container environments from one another.
Class Based Kernel Resource Management (2003)
An early framework that introduced the idea of classifying processes into groups or classes and assigning resources to each class as opposed to per-process resource limit. This concept played a significant role towards modern containers by influencing kernel features like cgroups, and namespaces which form the foundation of containers today.
Solaris Zones (AKA Solaris Containers) (2004)
At around the same time as the developments in Linux, Solaris, a UNIX operating system introduced Zones, also known as Solaris Containers that isolated processes into groups that could only interact with processes within the same group.
Process Containers (2006/2007)
Google used a system called Borg(precursor to Kubernetes) to manage its massive infrastructure. Borg, developed in the early 2000s allowed Google to run many tasks across thousands of machines in isolated containers. They needed an efficient way to manage, control and isolate resources, so around 2006, Google engineers began developing process containers, an early implementation of resource isolation and control for running workloads efficiently on shared infrastructure.
This work on process containers evolved and was later renamed cgroups( Control Groups) when it became a part of the Linux kernel in 2007. cgroups allowed fine-grained control over CPU, memory, I/O and other resources at the group level. This “class-based” or group-based management approach ensured that resources in one group did not starve resources in another group or class. Cgroups became the foundation for Docker, Kubernetes and modern container runtimes like containerd and CRI-O.
Additional Linux Kernel Namespaces (2006-2013)
Between 2006 and 2013, the Linux kernel introduced additional namespaces that allowed isolation in process IDs, inter-process communication, network stacks and user IDs. User namespaces were the last namespace to be added. They allow unprivileged users to create a namespace or isolated environment that allows running privileged processes within the namespace while disallowing running them outside the environment. This enables a process to have root-like privileges within its own namespace without actually having those privileges outside of it in the host.
The Modern Container Ecosystem (2008-Present)
- LXC (2008) – One of the first Linux container implementations, providing a userspace interface for managing containers.
- Docker (2013) – Popularised container usage with an easy-to-use interface, standardized images and an ecosystem that simplified deployment.
- Container Orchestration (2014-2015)
- Docker Swarm (2014) – Docker began work on Swarm, an orchestration tool for managing containerised applications
- Kubernetes (2015) – Google built and open-sourced Kubernetes in 2015, it is now the de facto standard for orchestrating large-scale containerised applications.
- LXD (2016) – Developed by Canonical, LXD extended LXC by introducing a more powerful container management interface with better security and networking features.
Security Concerns
Despite their advantages, VMs and containers are not immune to vulnerabilities. In 2018, critical flaws like Spectre and Meltdown exposed security risks in both technologies, emphasizing the need for ongoing improvements.
Conclusion
I enjoyed looking into the history of VMs and containers and learning how the technology behind them developed at the same time. Let me know in the comments how you use containers or virtual machines. Thanks for reading.
Resources
If you’re interested in learning more about the history of VMs and contaiers, consider looking at the resources listed below: