SSD vs NVMe vs HDD: Server Storage Performance Compared

The storage behind a server often limits performance more than the CPU: databases, package installs, container image pulls and backups all wait on the disk. Servers today use three kinds of drives: spinning hard disks (HDD), SATA solid state drives (SSD) and NVMe SSDs connected over PCIe. This guide explains how they differ, which workloads fit each one, and how to identify, benchmark and monitor the disks in a Linux server running Ubuntu 24.04.

How the three technologies differ

• HDD stores data on spinning magnetic platters read by a moving head. Every random access has to wait for the head to move and the platter to rotate, which takes several milliseconds. Sequential reads are reasonable; random I/O is slow.
• SATA SSD stores data in NAND flash with no moving parts, so random access is fast. It uses the SATA interface and the AHCI protocol designed for hard disks, which caps throughput at about 550 MB/s and allows a single command queue of 32 commands.
• NVMe SSD uses the same kind of flash but connects directly over PCIe with the NVMe protocol, which supports up to 65,535 queues of up to 65,536 commands each. Throughput scales with the PCIe generation, and latency is lower because there is no SATA controller in the path.

Quick comparison

Typical figures for current enterprise and data center drives. Individual models vary, so check the datasheet of the exact drive.

The key number for servers is usually random IOPS and latency, not sequential throughput. A database, a mail server or a busy web application does small random reads and writes all day, which is exactly where HDDs are hundreds of times slower than flash.

Endurance and reliability

Flash cells wear out after a finite number of writes, so SSDs are rated for endurance:

• TBW (terabytes written): total data that can be written over the warranty period.
• DWPD (drive writes per day): how many times the full capacity can be written every day during the warranty. Read-intensive data center drives are usually rated around 1 DWPD, mixed-use drives around 3 DWPD.

Enterprise SSDs also include power loss protection (capacitors that flush the write cache on power failure), which consumer drives often lack. For servers, prefer data center drives over consumer models even at similar speeds.

HDDs do not have a write limit, but their mechanical parts fail over time, and rebuilding a large failed disk in a RAID array can take a day or more. Whatever the drive type, redundancy (RAID or replication) protects against a drive failure and backups protect against everything else.

Identifying the disks in your server

List the block devices, whether they are rotational and the transport they use:

lsblk -d -o NAME,SIZE,ROTA,TRAN,MODEL

NAME      SIZE ROTA TRAN   MODEL
sda       3.6T    1 sata   ST4000NM000A-2HZ100
sdb     960.2G    0 sata   SAMSUNG MZ7LH960HAJR-00005
nvme0n1   1.7T    0 nvme   SAMSUNG MZQL21T9HCJR-00A07

ROTA is 1 for a spinning disk and 0 for flash. Devices named nvme* are NVMe drives.

Note

On a virtual server (VPS), disks are presented by the hypervisor as virtual devices such as vda with no transport shown, and ROTA does not reliably reflect the physical storage underneath. Check your provider's plan details for the storage type, and use a benchmark to measure real performance.

For NVMe drives, install nvme-cli to see model, firmware and usage:

sudo apt install nvme-cli
sudo nvme list

Benchmarking with fio

fio is the standard tool for measuring disk performance. It can test random and sequential I/O with realistic queue depths. Install it:

sudo apt install fio

Create a working directory on the filesystem you want to test. The tests below write a 2 GB file there, never to the raw device, so they do not destroy data:

sudo mkdir -p /var/tmp/fio
cd /var/tmp/fio

Measure random 4K read IOPS, the most important number for databases and general server workloads:

sudo fio --name=randread --filename=/var/tmp/fio/testfile --size=2G \
  --rw=randread --bs=4k --iodepth=32 --numjobs=4 --ioengine=libaio \
  --direct=1 --runtime=60 --time_based --group_reporting

The summary shows IOPS, bandwidth and latency:

randread: (groupid=0, jobs=4): err= 0: pid=4121: ...
  read: IOPS=412k, BW=1609MiB/s (1687MB/s)(94.3GiB/60001msec)
    clat (usec): min=18, max=2150, avg=309.40, stdev=95.12

Measure random 4K writes the same way with --rw=randwrite, and sequential throughput with large blocks:

sudo fio --name=seqread --filename=/var/tmp/fio/testfile --size=2G \
  --rw=read --bs=1M --iodepth=16 --numjobs=1 --ioengine=libaio \
  --direct=1 --runtime=60 --time_based --group_reporting

Remove the test file when you finish:

sudo rm -rf /var/tmp/fio

As a rough guide, an HDD returns a few hundred random read IOPS, a SATA SSD tens of thousands, and a local NVMe drive hundreds of thousands. On shared or network storage the results also depend on the platform's limits per volume. --direct=1 bypasses the page cache so you measure the disk and not the RAM.

Monitoring health

Install smartmontools to read the drive's health counters:

sudo apt install smartmontools
sudo smartctl -a /dev/nvme0

On an NVMe drive, the most useful lines are:

Critical Warning:                   0x00
Temperature:                        38 Celsius
Available Spare:                    100%
Percentage Used:                    3%
Data Units Written:                 91,512,334 [46.8 TB]
Media and Data Integrity Errors:    0

Percentage Used estimates how much of the rated endurance has been consumed; Critical Warning must be 0x00. For SATA drives use sudo smartctl -a /dev/sda and watch Reallocated_Sector_Ct, Current_Pending_Sector and, on SSDs, the wear leveling attribute.

The smartmontools package also installs the smartd daemon, which checks drives periodically and logs problems. Confirm it is running:

systemctl status smartmontools --no-pager

This only works on physical servers with direct access to the drives; virtual disks on a VPS do not expose SMART data.

Keeping SSDs and NVMe drives fast

Two defaults on Ubuntu 24.04 already cover the important settings for flash storage.

TRIM tells the drive which blocks are no longer in use, so it can keep write performance high. Ubuntu runs it weekly through a systemd timer. Check that it is enabled:

systemctl status fstrim.timer --no-pager

Run it once manually to confirm the filesystems support it:

sudo fstrim -av

/boot: 1.2 GiB (1293418496 bytes) trimmed on /dev/nvme0n1p2
/: 1.6 TiB (1754112450560 bytes) trimmed on /dev/nvme0n1p3

The weekly timer is preferred over the discard mount option, which issues TRIM on every delete and can slow down some drives.

I/O scheduler: NVMe drives work best without a scheduler. Check it:

cat /sys/block/nvme0n1/queue/scheduler

[none] mq-deadline

The value in brackets is active. none is the default for NVMe on Ubuntu and should be left as is; SATA SSDs and HDDs typically use mq-deadline.

Use cases

HDD is the right choice for:

• Backup targets and archive storage, where cost per TB matters more than speed.
• Large media libraries and object storage with mostly sequential access.
• Log retention and cold data that is written once and rarely read.

SATA SSD fits:

• Boot and operating system drives on servers with no NVMe slots.
• Web servers, small databases and general-purpose virtual machines on older hardware.
• Upgrading existing servers with SATA bays, where it delivers most of the practical improvement over HDD.

NVMe SSD is the best choice for:

• Databases (MySQL, PostgreSQL, ClickHouse) and anything sensitive to random I/O latency.
• Virtualization hosts running many VMs, and container hosts with many images.
• Build servers, CI runners, search indexes and caches.
• Any new server where the budget allows it, since the price difference with SATA SSD has become small.

A common layout on dedicated servers combines both worlds: NVMe drives in RAID 1 or RAID 10 for the system, databases and applications, and large HDDs for backups and bulk storage.

Which one should you choose?

• If the workload is interactive (websites, APIs, databases, VMs), choose NVMe, or SATA SSD where NVMe is not available. Do not run databases on HDD.
• If the workload stores large amounts of data that is rarely accessed or read sequentially, choose HDD for its cost per TB.
• If you are unsure, benchmark your real workload: the IOPS and latency from fio tell you more than the drive type on a spec sheet.

Conclusion

HDDs remain the cheapest way to store large volumes of cold data, SATA SSDs removed the random I/O bottleneck of spinning disks, and NVMe drives push throughput and latency much further for databases and virtualization. Identify your disks with lsblk, measure them with fio and monitor their health with smartctl. As next steps, set up RAID for redundancy on dedicated servers, and make sure your backups live on a different device than your data.