RAG golden set: extracting benchmark questions with an LLM
How we turned 33 SEC filings into 40 benchmark questions that genuinely require combining documents. The model proposed 214; ten machine checks…
Tag
Tag: benchmark
RAG on SEC filings: anatomy of a benchmark corpus
Before comparing seven RAG frameworks, we measured the data they will face. 33 NVIDIA filings, 749,859 tokens, and eight repeatable patterns of…
NVMe vs SATA LLM model loading: 10 runs per drive, measured
A 19.92 GiB Qwen model loaded 2.25x faster from NVMe than SATA, yet time to first token was identical. Cold-start benchmark with…
Benchmark lab for local models: real 27B-35B results on one 48 GB GPU
A practical benchmark lab for local models on AMD PRO W7900 48 GB. Real tok/s, TTFT, disk size, VRAM after loading and…
Multi-Token Prediction on AMD W7900: 1.85× speedup with llama-server Vulkan (Qwen3.6-35B benchmark)
We benchmark Multi-Token Prediction in llama-server (llama.cpp, Vulkan) on AMD PRO W7900. Real results: 49.7 → 91.8 tok/s, up to 2× on…