The Cloud Compute Stack: VMs, Containers, Kubernetes, and Serverless Explained

Modern cloud applications are built on multiple layers of abstraction. Much like cloud storage evolved from physical disks to distributed object storage, cloud compute has progressed from dedicated physical servers to highly abstracted execution environments that allow developers to deploy code without managing infrastructure.

Each layer was introduced to solve limitations of the one beneath it. None of these technologies replace their predecessors entirely. Instead, they optimize for different workloads, operational models, and business requirements.

Understanding where each layer fits-and where it does not-is critical when designing scalable, secure, and maintainable systems.

The Compute Stack

A simplified view of the compute stack looks like this:

Applications
│
├── Serverless Functions
│
├── Kubernetes
│
├── Containers
│
├── Virtual Machines
│
├── Hypervisor
│
└── Physical Hardware

Unlike storage, where data generally persists independently of applications, compute is fundamentally about executing workloads. Every higher abstraction attempts to improve resource utilization, deployment speed, scalability, and operational efficiency while introducing new tradeoffs.

Layer 1 - Physical Infrastructure

Everything begins with physical hardware.

A modern server consists of:

• CPUs
• Memory
• Storage devices
• Network interfaces
• Power supplies
• Hardware security modules
• Management controllers

Large cloud providers operate millions of these machines across geographically distributed data centers.

A single physical server is extremely powerful but presents several limitations:

• One operating system per machine
• Poor resource utilization
• Difficult scaling
• Hardware failures directly impact workloads
• Long provisioning times

Historically, many enterprise servers averaged less than 20% CPU utilization because applications were deployed one per server to avoid conflicts.

The obvious question became:

How can multiple workloads safely share the same hardware?

That led to virtualization.

Layer 2 - Hypervisors

A hypervisor is software that allows multiple operating systems to run simultaneously on the same physical machine.

Instead of dedicating one server to one application, the hypervisor partitions hardware into isolated virtual machines.

Application
Operating System
Virtual Hardware
───────────────
Hypervisor
───────────────
Physical Hardware

The hypervisor manages:

• CPU scheduling
• Memory allocation
• Virtual networking
• Virtual disks
• Hardware isolation

Two primary categories exist.

Type 1 Hypervisors

Run directly on hardware.

Examples include:

• VMware ESXi
• Microsoft Hyper-V
• Xen
• KVM

These dominate enterprise infrastructure because they provide strong isolation and high performance.

Type 2 Hypervisors

Run on top of another operating system.

Examples include:

• VirtualBox
• VMware Workstation

These are primarily used for development and testing.

Why Virtualization Changed Everything

Virtualization dramatically increased infrastructure efficiency.

Instead of:

Server A → Application A
Server B → Application B
Server C → Application C

Organizations could consolidate workloads:

Single Physical Server

├── VM A
├── VM B
├── VM C
├── VM D
└── VM E

Benefits included:

• Better hardware utilization
• Reduced power consumption
• Easier backups
• Live migration
• Faster provisioning
• Disaster recovery

Cloud computing would not exist without virtualization.

Layer 3 - Virtual Machines

A virtual machine contains everything required to run an operating system.

Application

Libraries

Guest Operating System

Virtual Hardware

Each VM behaves like an independent computer.

Advantages include:

• Strong isolation
• Different operating systems on one host
• Mature tooling
• Stable environments
• Easy migration between hosts

However, every VM includes a complete operating system.

That means:

• Hundreds of megabytes to several gigabytes of overhead
• Longer boot times
• Higher memory consumption
• Slower deployments

As applications became more distributed, these costs became increasingly noticeable.

Why Containers Emerged

Developers encountered a recurring challenge:

“The application works on my machine.”

Different operating systems, package versions, dependencies, and runtime environments frequently caused deployments to fail.

Virtual machines solved isolation but were often too heavy.

Containers approached the problem differently.

Instead of virtualizing hardware, they virtualize the operating system.

Layer 4 - Containers

Containers package:

• Application code
• Runtime
• Libraries
• Dependencies
• Configuration

Unlike virtual machines, containers share the host operating system kernel.

Application A

Application B

Application C

Shared Operating System Kernel

Physical Machine

Because there is no separate guest operating system, containers:

• Start in seconds or milliseconds
• Consume less memory
• Require less storage
• Allow much higher density

A server that runs dozens of virtual machines can often run hundreds of containers.

Docker Standardized Containers

Container technology existed before Docker.

Technologies like:

• LXC
• Solaris Zones
• FreeBSD Jails

had already demonstrated operating system-level isolation.

Docker simplified adoption by standardizing:

• Image format
• Packaging
• Networking
• Distribution
• Build process

A Docker image became a portable unit of deployment.

FROM python:3.12

COPY app.py .

RUN pip install flask

CMD ["python","app.py"]

The resulting image behaves consistently across development, testing, and production.

Containers Are Not Virtual Machines

A common misconception is that containers replace VMs.

They solve different problems.

Many production systems run containers inside virtual machines rather than directly on physical hardware.

The Operational Challenge

Containers introduced a new problem.

Running one container is easy.

Running thousands is not.

Questions quickly emerge:

• Which server should run a container?
• What happens if a node fails?
• How are updates rolled out?
• How is networking managed?
• How are secrets distributed?
• How is load balancing performed?

This is orchestration.

Layer 5 - Kubernetes

Kubernetes is a container orchestration platform.

Its primary responsibility is maintaining the desired state of an application.

Instead of telling infrastructure:

Start container A on machine X.

You declare:

I want:

10 application instances

Always available

Automatically restarted

Load balanced

Securely networked

Kubernetes continuously works to achieve that state.

Core Kubernetes Concepts

A simplified architecture includes:

Cluster

├── Control Plane
│
└── Worker Nodes
     ├── Pod
     ├── Pod
     ├── Pod
     └── Pod

Important building blocks include:

• Pods
• Deployments
• Services
• Ingress
• StatefulSets
• DaemonSets
• ConfigMaps
• Secrets
• Persistent Volumes

Each addresses a specific operational concern.

Why Kubernetes Became Popular

Large organizations needed:

• Automatic scheduling
• Self-healing
• Rolling deployments
• Service discovery
• Resource limits
• Horizontal scaling

Without orchestration, operating hundreds of containers becomes operationally expensive.

Kubernetes automates many repetitive infrastructure tasks.

Kubernetes Tradeoffs

Kubernetes is powerful but introduces complexity.

Engineers must understand:

• Networking
• DNS
• Service meshes
• Storage integration
• Scheduling
• Security policies
• Certificates
• Observability
• Cluster upgrades

For smaller systems, Kubernetes may introduce more operational overhead than it eliminates.

Not every application requires container orchestration.

Layer 6 - Serverless Computing

The next abstraction removed infrastructure management almost entirely.

Instead of deploying containers, developers deploy functions.

Incoming Request

↓

Function Executes

↓

Function Terminates

Infrastructure provisioning, scaling, and lifecycle management become platform responsibilities.

Why Serverless Emerged

Many workloads are event-driven.

Examples include:

• Image processing
• File uploads
• Notification systems
• Scheduled tasks
• API endpoints
• Data transformation
• Background jobs

Keeping virtual machines running continuously for infrequent workloads wastes resources.

Serverless platforms allocate compute only when required.

Benefits

Serverless offers:

• Zero server management
• Automatic scaling
• Pay-per-execution pricing
• Fast deployment
• Built-in fault tolerance

For sporadic workloads, utilization can approach 100%.

Tradeoffs

Serverless is not universally suitable.

Challenges include:

• Cold starts
• Execution time limits
• Stateless execution
• Vendor-specific implementations
• Debugging complexity
• Limited runtime customization

Long-running services or latency-sensitive applications may be better served by containers or virtual machines.

Layer 7 - Edge Compute

Traditional cloud architectures execute workloads in centralized data centers.

However, applications increasingly require processing closer to users.

Examples include:

• Industrial automation
• Autonomous systems
• Retail analytics
• IoT
• Video processing
• Smart cities
• Content delivery

Sending every request to a distant cloud region introduces unnecessary latency.

Edge computing moves workloads closer to where data is generated.

Device

↓

Nearby Edge Node

↓

Regional Cloud

↓

Central Cloud

Only required data travels to centralized infrastructure.

Compute and Storage Work Together

Although compute and storage are separate architectural layers, they are tightly connected.

Examples include:

Applications rarely operate on compute alone.

Choosing one layer often influences storage architecture, backup strategies, and disaster recovery planning.

Security Considerations Across the Stack

Every abstraction changes the security model.

Physical Infrastructure

Focus areas include:

• Physical access
• Firmware integrity
• Hardware tampering
• Secure boot

Hypervisors

Security priorities include:

• VM isolation
• Hypervisor patching
• Escape vulnerabilities
• Secure management interfaces

Virtual Machines

Each VM maintains its own:

• Operating system
• Firewall
• User accounts
• Patch lifecycle

This increases administrative overhead but provides strong isolation.

Containers

Containers introduce new concerns:

• Image integrity
• Supply chain attacks
• Vulnerable base images
• Runtime privileges
• Shared kernel exposure

Least privilege and image signing become increasingly important.

Kubernetes

Cluster security expands significantly.

Areas requiring attention include:

• RBAC
• Network Policies
• Admission Controllers
• Secrets management
• Pod Security standards
• API server security
• Node hardening

Misconfigurations often represent a greater risk than Kubernetes itself.

Serverless

Operational security improves because platform patching is largely managed.

However, developers remain responsible for:

• Identity management
• API security
• Secret handling
• Event validation
• Authorization logic

Application security does not disappear simply because infrastructure becomes abstracted.

Cost Considerations

Each layer shifts where costs are incurred.

The least expensive architecture depends on workload characteristics.

A continuously running service may be more economical on virtual machines or containers.

An infrequently triggered workload may be dramatically cheaper on serverless platforms.

Architecture decisions should be driven by utilization rather than trends.

Choosing the Right Layer

There is no universally superior compute platform.

Each addresses different operational priorities.

Modern architectures often combine multiple layers rather than standardizing on one.

A common production environment might include:

• Virtual machines hosting Kubernetes worker nodes
• Containers running application services
• Serverless functions processing asynchronous events
• Edge nodes handling latency-sensitive requests
• Object storage for application assets
• Block storage backing databases

The stack is layered because each abstraction solves a different class of problems.

Final Thoughts

The evolution of cloud compute has been driven by one consistent objective: increasing efficiency while reducing operational burden.

Virtualization enabled efficient hardware utilization. Containers improved portability and deployment speed. Kubernetes automated large-scale operations. Serverless minimized infrastructure management. Edge computing brought execution closer to users.

None of these technologies obsolete the others. Instead, they form a layered ecosystem where each abstraction offers distinct advantages depending on workload characteristics, operational maturity, security requirements, and cost constraints.

Architects should resist the temptation to adopt the highest abstraction by default. The best compute platform is the one that matches the needs of the application-not the one with the most features. A clear understanding of the entire compute stack enables informed decisions that balance performance, complexity, security, and long-term maintainability.