Part of Running LLMs on Your Own Hardware: From “Will It Even Fit” to a Model That Does Your Work — running large language models on your own hardware.
llama-server build 35c9b1f with the Vulkan backend · Qwen3.6-35B-A3B-UD-Q4_K_S (19.92 GiB) · AMD Radeon PRO W7900 48 GB · HP Z6 G4 · 10 runs per drive, file cache purged before every run.
Comparing NVMe vs SATA LLM model loading gives a result that splits cleanly in two. Loading a 19.92 GiB model took 25.74 seconds from NVMe and 57.97 seconds from SATA, a 2.25× difference. Time to first token, measured immediately afterwards, was 0.184 seconds on NVMe and 0.178 seconds on SATA — statistically identical. A faster drive buys you startup time, and nothing else.
- Cold model load: 25.74 ± 0.73 s on NVMe against 57.97 ± 0.74 s on SATA, 10 runs each.
- Time to first token was 0.184 s on NVMe and 0.178 s on SATA — the SATA drive was nominally faster, which shows the difference is noise.
- Raw sequential throughput differed by 5.07×, yet load time differed by only 2.25×, because roughly 14–17 s of the load is fixed work.
- Once weights sit in VRAM, storage leaves the critical path entirely; tokens per second depend on the GPU.
- The saving is 32.24 seconds per server start, which matters for frequent restarts and not at all for long sessions.
How NVMe vs SATA LLM model loading was measured
The question this benchmark answers is narrow on purpose. It asks how much a faster drive changes the experience of running a local language model, separating the part storage controls from the part it does not.
The hardware and the model
Both drives lived in the same workstation, so the CPU, memory and GPU were identical between runs. Only the file location changed.
| Component | Detail |
|---|---|
| Workstation | HP Z6 G4, 2 × Xeon Gold 5118, 64 GB RAM |
| GPU | AMD Radeon PRO W7900, 48 GB VRAM |
| Drive C: | Kingston KC3000 2 TB, NVMe PCIe 4.0 on a Gen3 x4 link |
| Drive D: | Intel D3-S4510 480 GB, SATA 6 Gb/s |
| Engine | llama-server build 35c9b1f, Vulkan backend |
| Model | Qwen3.6-35B-A3B-UD-Q4_K_S.gguf, 19.92 GiB |
| Settings | -ngl 99 -c 4096, full GPU offload |
The model file was copied byte-for-byte to the second drive and both copies verified identical at 21,388,319,008 bytes. Consequently any difference in timing came from the storage path rather than the data.
Why the file cache had to be purged
This is where most storage benchmarks quietly fail. With 64 GB of RAM, a 19.92 GiB file stays cached after the first read, so every subsequent run measures memory bandwidth instead of the drive.
An early attempt showed exactly that. The SATA drive reported 2,723 MB/s, which exceeds the physical ceiling of a 6 Gb/s bus by a factor of five. Therefore each run in the final benchmark purges the standby list first, through NtSetSystemInformation with MemoryPurgeStandbyList. Every measurement below is a genuine cold start.
NVMe vs SATA LLM model loading: the cold-start times
The load timer starts when llama-server launches and stops when its /health endpoint first returns {"status":"ok"}. That window covers reading the file, parsing the GGUF, allocating VRAM and uploading the weights to the GPU.
| Drive | Mean load | Std. dev. | Range | Runs |
|---|---|---|---|---|
| Kingston KC3000 (NVMe) | 25.74 s | 0.73 s | 24.97–27.14 s | 10 |
| Intel D3-S4510 (SATA) | 57.97 s | 0.74 s | 57.00–59.06 s | 10 |
The spread is tight on both drives, at roughly 1.3% of the mean. As a result the 32.24-second gap is far larger than any run-to-run variation, and the comparison needs no statistical hedging.
Time to first token: where the NVMe vs SATA gap disappears
Immediately after each load, the same prompt went to /completion with streaming enabled. The timer stopped when the first chunk arrived. This is the metric users actually feel while working.
NVMe averaged 0.184 ± 0.006 s and SATA averaged 0.178 ± 0.010 s across the same runs. All 20 measurements fell between 0.159 and 0.195 seconds regardless of which drive held the file.
That outcome is not surprising once you think about where the weights are. With -ngl 99 the entire model sits in the 48 GB of VRAM, so generation reads from GPU memory. The drive is no longer in the path, and the small difference simply reflects scheduler noise.
Free ebook
Free AI Video, Generated Locally
Working scripts and measured benchmarks. Free.
No spam. Unsubscribe at any time.
Why 5× throughput becomes 2.25× in LLM model loading
Raw sequential reads of the same file, measured with unbuffered I/O, differ far more than the load times do.
The arithmetic explains the gap. At 2,339.6 MB/s the NVMe drive needs 8.90 seconds to read 19.92 GiB, while the SATA drive needs 44.33 seconds at 461.6 MB/s. Subtracting those from the measured load times leaves 16.8 seconds of non-read work on NVMe and 13.6 seconds on SATA.
That residue is the fixed cost of starting an inference server: parsing the GGUF header, allocating VRAM, uploading roughly 20 GB across PCIe, and initialising the Vulkan backend. None of it depends on the drive. Consequently a storage upgrade compresses only part of the total, and the headline throughput ratio overstates the practical gain.
This is worth remembering when reading drive reviews. A benchmark showing 7,000 MB/s describes the best case for a queue depth no single-threaded loader produces. Even here, the KC3000 reached 2,339.6 MB/s rather than its rating, partly because the Z6 G4 negotiates a PCIe 3.0 link and partly because sequential reads at queue depth one cannot saturate NVMe.
What NVMe vs SATA means for buying decisions
The practical value of a faster drive scales with how often you restart the model server, not with how much you use it.
Consider two working patterns. Someone who loads a model once each morning saves 32 seconds a day, which is irrelevant. Someone iterating on quantisations, switching between models or restarting after configuration changes might reload fifteen times in a session, and that becomes eight minutes of waiting removed.
Development workflows fall firmly in the second group. Comparing two models means loading both repeatedly. Testing a context-length change means a restart. Debugging a crash means another. Therefore the drive matters most exactly when you are experimenting, and least when you are simply using a stable setup.
There is a second argument that this benchmark does not measure directly. A 2 TB NVMe drive holds far more models than a 480 GB SATA drive, and capacity determines whether you can keep several models available at all. In practice that constraint bites sooner than load time does.
How the numbers behave over a working session
A single load time is easy to dismiss as half a minute. Across a session the arithmetic changes character, so it helps to project it onto realistic patterns of use.
| Working pattern | Loads per day | SATA waiting | NVMe waiting | Saved |
|---|---|---|---|---|
| Stable setup, one model | 1 | 58 s | 26 s | 32 s |
| Occasional switching | 5 | 4 min 50 s | 2 min 9 s | 2 min 41 s |
| Active development | 15 | 14 min 30 s | 6 min 26 s | 8 min 4 s |
| Automated evaluation | 40 | 38 min 39 s | 17 min 10 s | 21 min 29 s |
The bottom row is the interesting one, because automated evaluation harnesses restart the server between configurations by design. In that setting the storage upgrade removes over twenty minutes from a single sweep, which is no longer a rounding error.
Conversely the top row explains why many people upgrade and notice nothing. If your workflow loads a model once and keeps it resident for hours, the drive contributes 32 seconds to your entire day. Expectations set by drive marketing then collide with a workload that never touches storage after the first minute.
Limits of this benchmark
Several caveats deserve stating plainly, because they bound how far these numbers travel.
First, the workstation is a PCIe 3.0 platform. A current desktop with a Gen4 or Gen5 link would show a wider throughput gap and a shorter NVMe load time, though the fixed 14–17 second overhead would remain.
Second, this tests one model at one quantisation. A smaller 7B model loads in a few seconds from either drive, so the absolute saving shrinks proportionally. Conversely a 70B model roughly doubles it.
Third, full GPU offload is what makes the TTFT result so clean. If the model did not fit in VRAM and layers streamed from disk during generation, storage would re-enter the critical path and the conclusion would change completely.
Finally, these are cold-start numbers by design. In a session where the same model is reloaded without a reboot, the file may still sit in the page cache, and load times drop dramatically on both drives.
Frequently asked questions
Does an NVMe drive increase tokens per second?
No. Generation speed depends on the GPU and the model, not on storage. Time to first token measured 0.184 s on NVMe and 0.178 s on SATA in the same test.
How much faster is NVMe vs SATA for LLM model loading?
2.25× in this measurement: 25.74 seconds against 57.97 seconds for a 19.92 GiB model, averaged over 10 runs per drive.
Why is the load time gap smaller than the throughput gap?
Because 14 to 17 seconds of the load is fixed work — parsing, VRAM allocation, PCIe upload and backend initialisation. Only the file-reading portion benefits from a faster drive.
Is a PCIe 5.0 drive worth buying for local LLMs?
Rarely. The fixed overhead does not shrink, so halving an already short read time yields diminishing returns. Capacity is usually the better investment.
Does this apply to Ollama and LM Studio as well?
Yes, since both use llama.cpp underneath and load GGUF files the same way. The absolute numbers differ with model size, but the split between load time and generation speed holds.
How do I benchmark my own drives fairly?
Purge the file cache before every run, otherwise you measure RAM. On Windows that means MemoryPurgeStandbyList, or unbuffered reads with FILE_FLAG_NO_BUFFERING for raw throughput.
Would a RAM disk load models faster still?
It would remove the read time, leaving the fixed 14 to 17 seconds. However, dedicating 20 GB of RAM to hold a model you also copy into VRAM is rarely a good trade.
Summary
NVMe vs SATA LLM model loading comes down to one honest sentence: the drive changes how long you wait for the server to start, and nothing about how fast it answers. Loading a 19.92 GiB model took 25.74 ± 0.73 seconds from NVMe against 57.97 ± 0.74 seconds from SATA, while time to first token stayed at roughly 0.18 seconds on both.
Raw sequential throughput differed by 5.07×, but only 2.25× of that reached the user, because 14 to 17 seconds of every load is fixed work independent of storage. Buy the faster drive for capacity and for the 32 seconds saved on each restart — not for tokens per second, which it cannot improve. For the sizing and price side of the decision, see the guide to SSD storage for local AI, and for the migration itself, cloning Windows to an NVMe SSD.
Free ebook
Free AI Video, Generated Locally
Run Wan 2.1 in ComfyUI on your own GPU — the scripts I use, measured times, sample clips. No cloud, no API keys.
No spam. Unsubscribe at any time.
Leave a Reply