Bare Metal vs Virtualization: Performance Compared

Choosing between a bare metal server and a virtual machine is mostly a question of how much of the hardware your workload needs and how predictable its performance must be. Modern hypervisors such as KVM run CPU-bound code at close to native speed, but disk I/O, network latency, memory-heavy workloads and noisy neighbours can still make a difference. This guide explains where virtualization overhead comes from, which workloads notice it, and how to measure it yourself on Ubuntu 24.04 instead of relying on generic benchmark numbers.

At a glance

Where virtualization overhead comes from

Hardware-assisted virtualization (Intel VT-x, AMD-V) lets guest code run directly on the physical CPU. Overhead appears only at the points where the hypervisor has to step in:

• VM exits. Privileged operations, some interrupts and I/O accesses trap into the hypervisor. Each exit costs time; workloads with many system calls or interrupts pay more.
• Memory translation. Guest virtual addresses are translated to guest physical and then to host physical addresses (EPT on Intel, NPT on AMD). TLB misses are more expensive, which affects large, random memory access patterns such as databases and in-memory caches. Huge pages reduce this cost.
• I/O path. A disk write from a VM goes through the guest driver (virtio), the host's QEMU or vhost process, and then the host's filesystem or block layer. That adds latency per operation, which matters for small, synchronous writes (database commits) more than for large sequential transfers.
• Scheduling and contention. A vCPU is a thread on the host. If the host has more vCPUs than physical cores and neighbours are busy, your vCPU waits. Linux reports this as steal time.

How the workloads compare

• CPU-bound work (compilation, encoding, most application code): the difference between a KVM guest and bare metal on the same hardware is usually small, in the low single digits of percent, as long as the vCPUs are not contended.
• Memory-intensive work (large in-memory databases, analytics): the gap grows with working set size and random access, because of translation overhead and NUMA effects. Huge pages and pinning a VM to one NUMA node help.
• Disk-intensive work (OLTP databases, message queues): this is where bare metal wins most clearly, especially on latency and on small synchronous writes to local NVMe.
• Network-intensive work (load balancers, packet processing, high-frequency trading): virtio with vhost is fast enough for most web traffic, but the lowest and most consistent latency needs direct hardware access (bare metal, or SR-IOV/PCI passthrough).
• Latency-sensitive work (real-time audio, games, trading): the average may look fine on a VM, but tail latency (p99, p99.9) is where contention and VM exits show up.

Because the results depend on the hypervisor configuration, the hardware generation and how busy the host is, the only numbers that matter are the ones you measure on the machines you are comparing.

Prerequisites

• Two servers running Ubuntu 24.04 LTS that you want to compare, for example a CubePath VPS and a CubePath bare metal server with a similar CPU generation.
• A non-root user with sudo privileges on both.
• Around 10 GB of free disk space for the disk test.

Step 1 - Checking whether you are on a VM

systemd-detect-virt prints the hypervisor type, or none on bare metal:

systemd-detect-virt

kvm

lscpu shows the CPU model, core count and NUMA layout, which you need to compare like with like:

lscpu | grep -E 'Model name|^CPU\(s\)|Thread|NUMA node\(s\)|Hypervisor'

Model name:                           AMD EPYC 9454P 48-Core Processor
CPU(s):                               4
Thread(s) per core:                   1
Hypervisor vendor:                    KVM
NUMA node(s):                         1

Step 2 - Measuring steal time

Steal time is the share of time your vCPU was ready to run but the host gave the physical CPU to something else. On bare metal it is always zero. Watch it with vmstat while the server is under its normal load:

vmstat 5 5

procs -----------memory---------- ---swap-- -----io---- -system-- -------cpu-------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st gu
 1  0      0 3120448  40212 612744    0    0     1     9  120  210  3  1 96  0  0  0

The st column is steal time in percent. Sustained values above a few percent mean your VM is competing for CPU, and performance will vary with what the neighbours are doing. The wa column shows time spent waiting for I/O, a hint that storage is the bottleneck.

Step 3 - Benchmarking CPU and memory with sysbench

Install sysbench on both servers:

sudo apt update
sudo apt install sysbench

Run a CPU test using all available cores for 30 seconds:

sysbench cpu --threads="$(nproc)" --time=30 run

CPU speed:
    events per second:  9432.51

Compare events per second per core between the two machines. Run the test a few times at different hours on the VM: the spread between runs tells you as much as the average.

Then measure memory bandwidth:

sysbench memory --memory-block-size=1M --memory-total-size=20G --threads="$(nproc)" run

20480.00 MiB transferred (21342.17 MiB/sec)

Step 4 - Benchmarking disk I/O with fio

fio is the standard tool for storage benchmarks. Install it:

sudo apt install fio

Run a 4K random read and write test against a test file, bypassing the page cache so you measure the disk and not RAM:

fio --name=randrw --filename="$HOME/fio-test" --size=4G --direct=1 \
    --ioengine=libaio --rw=randrw --rwmixread=70 --bs=4k --iodepth=32 \
    --numjobs=4 --runtime=60 --time_based --group_reporting

  read: IOPS=61.2k, BW=239MiB/s (251MB/s)(14.0GiB/60001msec)
    clat percentiles (usec):
     |  99.00th=[ 1004], 99.90th=[ 1942]
  write: IOPS=26.2k, BW=102MiB/s (107MB/s)(6143MiB/60001msec)

Look at IOPS and, above all, at the 99th and 99.9th percentile completion latency (clat percentiles). Bare metal NVMe typically shows much lower tail latency than virtualized storage.

To see the cost of synchronous writes, which is what a database commit does, run a single-threaded test that syncs after every write:

fio --name=syncwrite --filename="$HOME/fio-test" --size=1G --rw=randwrite \
    --bs=4k --iodepth=1 --numjobs=1 --fsync=1 --runtime=30 --time_based

Remove the test file when you are done:

rm "$HOME/fio-test"

Step 5 - Benchmarking the network with iperf3

Install iperf3 on both servers:

sudo apt install iperf3

Answer No if the installer asks whether to start iperf3 as a daemon. On the server you want to test, start it in server mode and temporarily allow its port:

sudo ufw allow from your_client_ip to any port 5201 proto tcp
iperf3 -s

From the other machine, run a 30-second test with four parallel streams:

iperf3 -c your_server_ip -P 4 -t 30

[SUM]   0.00-30.00  sec  32.8 GBytes  9.39 Gbits/sec   12   sender

For latency, ping -c 100 your_server_ip gives you the average and the maximum round trip, and mtr shows where delay appears on the path. Stop iperf3 with Ctrl+C and remove the firewall rule afterwards with sudo ufw delete allow from your_client_ip to any port 5201 proto tcp.

Reducing overhead when you stay virtualized

If the numbers show that a VM is good enough, these measures help keep it that way:

• Pick the right size. Many performance complaints are simply undersized VMs. Check vmstat for run queue (r) above the vCPU count and swap activity (si/so).
• Keep the OS and kernel current. virtio drivers and schedulers improve with each release.
• Use the platform's disk and network drivers. On KVM make sure the guest uses virtio (lsblk shows vda or sda on virtio-scsi, and ethtool -i eth0 shows driver: virtio_net).
• Tune the application, not only the VM. Database buffer sizes, connection pooling and caching usually yield more than any hypervisor tweak.

On bare metal, you also control the hardware settings. For consistent latency, use the performance CPU frequency governor, which cpupower from the linux-tools-common and linux-tools-$(uname -r) packages can set:

sudo apt install linux-tools-common "linux-tools-$(uname -r)"
sudo cpupower frequency-set -g performance

This setting does not persist after a reboot. It also has no effect inside most VMs, where the host controls CPU frequency.

Which one should you choose?

Choose bare metal when:

• The workload needs consistent low latency or predictable tail latency (trading, real-time media, game servers).
• You run I/O-heavy databases where local NVMe latency is the bottleneck.
• You need the whole machine anyway: large memory footprints, GPUs, many cores, or you want to run your own hypervisor (Proxmox VE, KVM) or Kubernetes nodes without a second virtualization layer.
• Licensing is per physical server, or compliance requires single-tenant hardware.

Choose virtual machines when:

• The workload fits comfortably in a few vCPUs and GBs of RAM, which is true for most web applications, APIs and small databases.
• You need to scale up or down quickly, take snapshots, or create environments on demand.
• Cost matters more than the last few percent of performance.

Combine both when it makes sense: run steady, heavy components (primary database, cache, storage) on bare metal and elastic or small services on VMs, connected through a private network.

Conclusion

Virtualization overhead on a modern KVM host is small for CPU-bound work and becomes noticeable for disk latency, memory-heavy workloads and anything sensitive to tail latency or noisy neighbours. Measure before deciding: run the same sysbench, fio and iperf3 tests on both options, repeat them at different times, and compare the variance as well as the averages. As next steps, benchmark your real application with a load testing tool, monitor steal time and I/O wait in production, and revisit the choice when your workload grows.