Part of RAG & Vector Databases — retrieval-augmented generation and vector search.
This RAGFlow benchmark measures a product rather than a library. RAGFlow ships a web interface, a task queue and four stateful services behind it, and it indexed 33 NVIDIA SEC filings in 103 seconds without calling a generative model once. Then I built its knowledge graph, which took 5.75 hours, and asked the same 40 cross-document questions. Then I built RAPTOR, which took 3.23 hours. The interesting part of the answer is not the score. It is that neither structure is reachable through the setting that claims to switch it on.
Key takeaways from this RAGFlow benchmark
- RAGFlow’s best configuration reached 67.5% strict document recall and 32.5% correct answers (13 of 40). The benchmark’s own difficulty floor — a 200-line naive retriever — sits at 70.0% and 15.0%.
- RAPTOR is the configuration to build. It cost 3.23 hours and moved the shipped default from 9 to 13 correct answers out of 40, with the highest answer-or-refuse accuracy in this benchmark, 85.0%. The knowledge graph cost 5.75 hours and moved it from 9 to 10.
- Building the same configuration twice does not give the same index. A second RAPTOR build on a clean stack changed the model’s prompt in 37 of the 40 questions, and moved recall to 72.5% while correct answers went the other way, to 11 of 40. The refusal accuracy and the zero invented numbers did not move at all.
- Neither structure is a retrieval mode. Turning on the graph mode, and turning on RAPTOR, each gave the model identical context in 40 questions out of 40 against the plain default on the same index. Both help by sitting in the index, not by being requested.
- RAGFlow scores zero on event-to-consequence questions in every mode. All five pair a three-chunk 8-K with a ninety-six-chunk 10-Q, and the short filing never reaches the prompt.
- Six retrieved chunks reach a median of four distinct documents, and in 28 of 40 questions one filing takes half the prompt or more. A budget counted in chunks is not a budget counted in documents.
What RAGFlow does differently from a RAG library
RAGFlow is not one container. It is a web server and a task executor in front of four stateful services: Elasticsearch for chunks and BM25, MySQL for metadata, MinIO for the original files, and Redis for the task queue. In practice, nothing runs in the same process as anything else.
That shape has one immediate consequence for the bill. Indexing calls no large language model at all. Instead it splits text into chunks, embeds each chunk, and writes both to Elasticsearch. The embedding model is a model, but not a generative one — it turns existing text into numbers and writes nothing new. Hence 103 seconds, against the hours its own knowledge graph costs later.
Moreover, the shape brings a problem that shaped how I ran this whole episode. RAGFlow writes its knowledge graph into the same Elasticsearch index it searches for chunks. Therefore a graph is not a mode you switch on beside the plain index. Instead it changes the thing you search, and unticking a box does not undo it.
The corpus itself never changes between episodes. Every filing comes from NVIDIA’s EDGAR archive, downloaded once and never edited. Forty questions carry a knowledge dispersion pattern and the number of filings needed to answer them. No question can be answered from a single passage, and five have no answer in the corpus at all, so a system that never refuses cannot score well.
The RAGFlow benchmark setup, command by command
This machine has no compose provider, so the stack starts with plain podman run. As a result every setting sits in one place instead of behind three layers of defaulted environment variables.
podman network create nvda-ragflow
podman run -d --name nvda-rf-es --network nvda-ragflow -p 1200:9200
-e discovery.type=single-node -e ELASTIC_PASSWORD=infini_rag_flow
-e "ES_JAVA_OPTS=-Xms2g -Xmx2g"
-v nvda-rf-esdata:/usr/share/elasticsearch/data
docker.io/library/elasticsearch:8.11.3
podman run -d --name nvda-rf-mysql --network nvda-ragflow -p 3306:3306
-e MYSQL_ROOT_PASSWORD=infini_rag_flow
-v nvda-rf-mysql:/var/lib/mysql docker.io/library/mysql:8.0.40
podman run -d --name nvda-rf-redis --network nvda-ragflow docker.io/valkey/valkey:8
podman run -d --name nvda-rf-server --network nvda-ragflow -p 9080:80 -p 9380:9380
--add-host llmhost:192.168.64.1 -v nvda-rf-logs:/ragflow/logs
docker.io/infiniflow/ragflow:v0.27.0
Three details that each cost an afternoon
First, a relay carries the generation traffic over plain HTTP. My llama-server serves HTTPS with a certificate issued to a hostname no container can match, so a small bridge listens on 8084 and forwards to it. Embeddings already run on plain HTTP and need no relay.
Second, host.docker.internal does not point at Windows. Inside a podman WSL machine it resolves to the WSL bridge instead. Therefore the script reads the host address from the machine’s default route and injects it under a name of your own, here llmhost.
Third, and finally, declare the embedding model’s real context length. In practice RAGFlow batches text up to the number you declare. Declare more than the server actually gives, and a batch is rejected in the middle of a build, hours in.
After that, one account, two model registrations and an API key, then the corpus goes in:
python articles/E-05-ragflow/bootstrap.py # account, models, API key
python articles/E-05-ragflow/run_scenario.py --scenario plain # corpus and base index
What the RAGFlow index cost
For comparability the dataset keeps RAGFlow’s defaults for the General chunk method: chunk_token_num 512, DeepDoc layout recognition, no auto-keywords, RAPTOR off, knowledge graph off.
| Measure | Value |
|---|---|
| Documents | 33 |
| Chunks | 1,797 |
| Wall-clock time for the whole index | 103 s |
Sum of per-document process_duration | 1,915 s |
| Peak resident memory, five containers | 8.07 GiB |
Those middle two rows disagree by a factor of nearly nineteen. The difference deserves a paragraph, because it is the kind of number that gets published by mistake.
All 33 documents start within 3.0 seconds of each other. The pipeline accepts several tasks at once, and a document’s clock starts when its task is accepted rather than when it is picked up. Consequently the reported duration includes the document’s own wait in the queue.
The clearest evidence is a pair of filings. An 8-K with three chunks “took” 88.6 s. A 10-Q with 106 chunks — thirty-five times the content — took 30.6 s. In other words, the 103 seconds is the cost of the index, and the per-document column is time spent in the pipeline rather than work done.

Above all, that figure is stable. The base index was rebuilt from scratch on four separate generations of the stack over two days, and every one produced the same 1,797 chunk rows.
What building the knowledge graph really measured
The default index calls no generative model. Its knowledge graph calls one on every chunk of every document, and that is where the hours go. It is also where this episode nearly published a wrong sentence about RAGFlow, so the failure earns more space than the success.
In the event, twenty-nine of the 33 documents came out of that build carrying the marker Document knowledge compilation done, with task status DONE and progress 1.0. Every check that reads a status field was satisfied. Then I counted the rows each document had actually written, and the shape made no sense as work: six documents stopped inside a 3.2-second spread of one another, with 12, 13 and 14 batches of work between them. Jobs of different sizes do not finish in the same second. That is a limit, not completion.

The arithmetic sits in two lines of RAGFlow’s model client:
timeout = int(os.environ.get("LLM_TIMEOUT_SECONDS", 600))
self.max_retries = kwargs.get("max_retries", int(os.environ.get("LLM_MAX_RETRIES", 5)))
600 seconds, six attempts, 3,600 seconds. The pipeline log carried 218 instances of openai.APITimeoutError: Request timed out.
Why a timeout punishes long documents and nothing else
The client starts its deadline when it sends a request, not when the server starts working on it. Knowledge compilation submits every batch of a document at once, and a llama-server with one slot serves them strictly one at a time. Therefore batch twelve is still queued when its 600 seconds expire, and the client hangs up before the model has read a token of it. Then it repeats that five more times.
By comparison, an 8-K is two to nine chunks. It fits in one batch, never waits behind anything, and always succeeded. A 10-Q is twelve to fourteen batches, and its tail never survived. A default client timeout is a silent filter on document length.
Meanwhile nothing in the run reported an error. The pipeline caught the exception exactly as it should, wrote an empty list of entities, logged a finished compilation and moved on.
Why this matters beyond RAGFlow
Consider what the benchmark would have printed. Graph mode would have scored badly on exactly the cross-document questions a graph is supposed to help with, and the write-up would have read: RAGFlow’s knowledge graph does not help when knowledge is spread across documents. Every number in that sentence would have been correctly measured. The sentence would still have been about an HTTP setting.
There is a cheap check that catches it, and it applies to any framework that builds a structure over documents with a model. Count the rows each document contributed, and confirm the longest documents contributed the most. If a hundred-chunk filing holds fewer entities than an eight-page press release, you are measuring a limit somewhere rather than a framework.
Fixing that takes one environment variable, LLM_TIMEOUT_SECONDS=7200. However, raising it was not enough to trust the result, for a second reason that took another day to find. A RAGFlow instance accumulates state between configurations, and can reproduce an earlier compilation in seconds while reporting it as finished work. The rows are real, the log is honest, and no work was done. So the instance was abandoned rather than repaired, and every measurement below comes from rebuilds on fresh containers.
RAGFlow benchmark results
Four columns, and the first thing to understand is that they are not four settings. They are two index configurations, each built on its own generation of the stack.

| Metric | vector (no structure) | hybrid (no structure) | hybrid (graph) | kg (graph) | raptor (summaries) |
|---|---|---|---|---|---|
| Strict document recall | 52.5% | 65.0% | 67.5% | 67.5% | 67.5% |
| Answers judged correct | 17.5% (7/40) | 22.5% (9/40) | 25.0% (10/40) | 25.0% (10/40) | 32.5% (13/40) |
| Numeric agreement | 40.0% of 20 | 41.7% of 24 | 48.1% of 27 | 50.0% of 26 | 46.9% of 32 |
| Answer-or-refuse calls right | 57.5% | 67.5% | 75.0% | 72.5% | 85.0% |
| Answers with invented numbers | 0 | 0 | 0 | 0 | 0 |
| Median latency | 11.54 s | 11.75 s | 11.53 s | 17.23 s | 12.82 s |
| Build cost on top of the index | — | — | 5.75 h | 5.75 h | 3.23 h |
Hybrid is the shipped default: one Elasticsearch query blending BM25 with dense vectors. Vector is the same index with the keyword weight at zero — the control that says how much of the result is the dense retriever. On this corpus the answer is blunt: turning keywords off costs 12.5 points of recall. Financial filings are full of exact strings, and a dense retriever alone gives them up.
Free ebook
Free AI Video, Generated Locally
Working scripts and measured benchmarks. Free.
No spam. Unsubscribe at any time.
Against the benchmark’s own floor
Every episode of this benchmark reports against the same floor, and the floor is deliberately crude: a 200-line script that embeds chunks and takes the top eight. It exists to calibrate how hard the forty questions are, not to compete. It retrieves 70.0% where RAGFlow’s best configuration retrieves 67.5%.
That is the useful reading of the floor. Retrieval on this corpus is not what separates a product from a script — both land near 70%. What separates them is downstream: RAGFlow answers 25.0% correctly against the floor’s 15.0%, and it is right about when to refuse twice as often, 75.0% against 37.5%. The floor refused 30 of 40 questions when only five have no answer in the corpus.
Neither system invented a number. That is worth stating plainly, because it is the failure mode readers fear most: zero fabricated figures across 160 answers.
RAGFlow benchmark retrieval: found, or shown to the model?
Split recall by how many filings a question needs, and the average stops being informative.

As a result, RAGFlow finds every piece of evidence for every question that lives in one filing. It finds both pieces for 35% of the questions that need two. That is not a ranking failure. It is the shape of the budget.
In practice the retrieval call scans about 1,024 candidates and then hands the model six chunks, per query, over the whole corpus. Asked instead for a deliberately wide page of fifty, the ranker had every evidence document somewhere in the list for 36 of the 40 questions. Of the 40:
- 27 had every evidence document in the prompt;
- 9 had them ranked inside the top 50 and cut before the prompt;
- 4 were never ranked at all.

That is nine against four. More of RAGFlow’s retrieval failures are the top-n cut than are search failures, and the two call for opposite fixes: one is solved by raising a number, the other by changing how you chunk or embed. Both counts come from a wide retrieval taken on the same index as the run.
Where the evidence sits in the ranking
Furthermore, the positions say the same thing from the other side. The first chunk of an evidence document appears at rank 2 in the median case — and 12 of 56 evidence documents first appear past the six chunks the model ever sees.
Meanwhile, this corpus makes that budget bite harder than most. Fourteen of the 33 filings hold three chunks or fewer, and they compete for the same six slots as a 197-chunk annual report.
The pattern this RAGFlow benchmark never scores on
One of the eight dispersion patterns is a flat zero in every mode. D5 is event-to-consequence: something is announced, and its effect appears in the next periodic report. RAGFlow retrieves the full evidence for none of the five, with the graph or without it.

In addition, the five questions share a shape. Each pairs an 8-K of three chunks with a 10-Q of ninety-six, and needs both.
| Mode | 10-Q in the prompt | 8-K in the prompt |
|---|---|---|
| vector (no graph) | 5 of 5 | 0 of 5 |
| hybrid (no graph) | 3 of 5 | 0 of 5 |
| hybrid (graph built) | 5 of 5 | 0 of 5 |
| kg (graph built) | 5 of 5 | 0 of 5 |
In every configuration the ranker finds the long document. It never shows the short one. Moreover the reason changes depending on which index you ask. On the index with no graph, the 8-K is in the ranked pool for three of the five questions and is cut before the six that reach the model. After the graph was compiled, it is in the pool for one: entity and relation rows compete for the same ranked positions as chunks, and a three-chunk filing loses them.
That is the bill for the graph that the headline numbers hide. Overall recall rose from 65.0% to 67.5% while the evidence for this pattern got harder to find.
By contrast, the crude floor retrieves both documents on all five. It spans six different documents in the median question against RAGFlow’s four, on a comparable number of chunks. In 28 of the 40 questions, a single filing takes half or more of RAGFlow’s six slots.
What the knowledge graph changed, and what it did not
Compiling the graph over all 33 documents cost 5.75 hours of model time and wrote 6,162 rows, with a further 2,781 in the dataset-level merge above them. Asking the same shipped default the same forty questions before and after:
- retrieval 65.0% → 67.5%;
- correct answers 9 → 10 out of 40;
- answer-or-refuse accuracy 67.5% → 75.0%;
- and 17 of the 40 questions retrieved a different set of documents.
That last line is the one to keep. Building the graph did not add a mode beside the existing one. It changed the existing one, because entity and relation rows live in the same index as the chunks and are ranked next to them. In the hybrid run, 16 of the 240 retrieved rows were graph rows, across 6 of the 40 questions. On the index without a graph, none.
The switch that changes nothing
RAGFlow has a retrieval setting, use_kg, which its documentation describes as querying the knowledge graph. On the index where the graph had been compiled, the default and the graph mode ran back to back over the same forty questions.
As it turned out, they returned identical context in 40 questions out of 40: the same six chunks, in the same order. Same retrieval, same 10 correct answers, same zero invented numbers. The only difference is 5.7 seconds more per question.
Consequently the whole benefit of the 5.75-hour compilation is already in the results before the switch is touched. It comes from the graph’s presence in the index, not from asking for it.
One company, two nodes in the RAGFlow benchmark graph
The corpus holds 33 filings by one company. Nevertheless RAGFlow’s merged graph keeps nvidia and nvidia corporation as two separate nodes, of degree 559 and 376, merged from 46 per-document rows. The next busiest node in the entire graph has degree 37.

Merging works on the normalised name, and entity resolution stays off by default, so the two strings never meet. A query that lands on one hub therefore cannot see the relations hanging off the other.
Merged entities forget when
In addition, the graph keeps two layers in one index: a row per entity per document, and a merged row carrying the list of documents it came from. Of the 987 merged entities, 489 span two or more documents, and 364 of those (74.4%) name no period at all — no year, no quarter, no “fiscal”.
micron, merged from eight filings: “A supplier of the company whose products were subject to sales restrictions announced by the Chinese government.”
In short: eight filings across three fiscal years, one sentence, no date. Moreover, merging joins descriptions on the entity’s name, and what distinguished them — the period — was never part of the name, so it had nowhere to survive. No model will pick the right year out of a description that names no year, and this corpus asks about a specific fiscal year constantly.
What RAPTOR changed in the RAGFlow benchmark, and what it cost
RAGFlow’s second structure is RAPTOR: it clusters a document’s chunks, writes a summary of each cluster with the model, clusters the summaries, and repeats until one node is left. The result is a tree of generated prose sitting in the same index as the filings.
Building it over the 33 documents took 3.23 hours — a little over half
what the knowledge graph cost. Counted from the index afterwards, it wrote 797 searchable
summaries, plus 33 tree structures that carry available_int: 0 and are
therefore invisible to retrieval.
82, 83, 51 Table of Contents, 17 Table of Contents. The largest was 20 characters against a median chunk of 1,864, so nothing that could answer a question was lost. Which step deletes them is still unestablished.
This time the structure does reach the model
The knowledge graph changed nothing a query is shown. RAPTOR is the opposite. Of the 240 chunks handed to the model across the forty questions, 99 were generated summaries — 41.2%. On five questions all six chunks were summaries, so the model answered without one sentence NVIDIA had actually filed. On seven it saw no summary at all.
That is the setup for a failure that did not happen. A summary is prose a model wrote about a filing, and it restates figures; a prompt made of summaries is a prompt of second-hand numbers. Yet RAPTOR invented no number in any of the forty answers, and it attempted more numeric questions than any other configuration — 32 against 27 — while posting the highest answer-or-refuse accuracy measured anywhere in this benchmark, 85.0%.
Four answers, not one
Because RAPTOR is a change to the index rather than a retrieval setting — more on that below — the honest comparison is the shipped default before and after the build:
The same hybrid mode, asked the same 40 questions | Correct | Build cost |
|---|---|---|
| on the plain index | 9 of 40 | — |
| after the knowledge graph | 10 of 40 | 5.75 h |
| after RAPTOR | 13 of 40 | 3.23 h |
Four correct answers for 3.23 hours, against the graph’s one for 5.75. Moreover the four are spread across four different dispersion patterns rather than concentrated in one, which is what you would hope for from a structure that summarises whole documents. A second build of the same configuration put that gain at two rather than four, which is measured below — the direction holds, the size is worth one build’s uncertainty.
The second switch that does nothing either
The adapter configures raptor exactly as it configures hybrid. So
hybrid was run again on the same stack, and the two runs are the same run:
- 40 of 40 questions retrieved identical documents;
- 40 of 40 received identical context chunks;
- every score identical — 67.5% recall, 13 correct, 85.0% refusal accuracy, 46.9% numeric;
- medians 12.82 s and 12.78 s.
Twice now, in the same product: a structure that helps by being in the index, and a mode switch that changes nothing. Neither of RAGFlow’s structures is a retrieval mode. Both are changes to the corpus, and the only way to measure either is to build a separate index and compare the default mode against itself.
Built twice: the same configuration, a different index
Every number above comes from one build. RAPTOR is the configuration this episode leads with, so it was built a second time, on a fourth clean stack, from the same 33 files with the same settings — and with a snapshot of the base row ids taken before the build, which is what identified the deleted page furniture above.
In fact, the base index reproduced exactly: 1,797 chunks for the fourth time, checked document by document rather than by total. Everything the model wrote did not. The second build produced 793 summaries instead of 797, different text, and a different spread across the tree layers. Comparing the two runs question by question:
The same raptor configuration, built twice | Build 1 | Build 2 |
|---|---|---|
| Strict document recall | 67.5% (27/40) | 72.5% (29/40) |
| Correct answers | 13 of 40 | 11 of 40 |
| Figures matched | 46.9% of 32 | 51.6% of 31 |
| Answer-or-refuse accuracy | 85.0% | 85.0% |
| Invented numbers | 0 | 0 |
| Median latency | 12.82 s | 13.13 s |
| Questions whose six chunks were identical | 3 of 40 | |
The prompt changed in 37 of 40 questions, and the aggregate moved by two. Specifically, three questions gained their full evidence and one lost it, for a net of plus two on recall. The judge changed its verdict on 11 questions, four of them out of CORRECT. So recall went up while correct answers went down, from the same rebuild.
More evidence retrieved is not more answers right. A summary competes for one of the six slots with the filing text, so the build that added a gold document to three prompts pushed the chunk carrying the figure out of others.
Which numbers survive a rebuild, and which do not
Two things did not move at all: the 85.0% answer-or-refuse accuracy, to the decimal, and the zero invented numbers. That split is the useful part. Claims about this system’s character — it refuses well, it does not make figures up — survived a rebuild. Claims about its count did not.
Contrast that with the plain index, which was also built twice, on two separate installations: 65.0% and 65.0%, the same 41.7% of 24 figures, the same everything. The difference between the two repeats is one thing. There is no generative model inside a plain build. Chunking, embedding, hybrid search and the top-n cut are deterministic across installs at temperature 0; a build in which a model writes summaries is not, and no setting makes it so, because clustering gets different input the moment one summary below it changes.
What to do with this. Therefore read the figures in this article as one build’s measurement, not as a constant of the framework. The spread from rebuilding alone is about two questions in forty on both headline axes — so a gap of that size between two systems, each built once, is not evidence that one is better. It is the reason the number quoted here stays attached to the build it came from.
RAGFlow benchmark pitfalls and limits
Finally, four traps cost real time here, and all four report success while producing nothing.
A stopped stack keeps building when you start it again. After a host reboot the index grew on its own: three dataset-merge tasks queued during the previous day’s build had never been acknowledged, so the start redelivered them. A repeat run settled what that cost — 40/40 identical chunks and answers, recall unmoved — but a stack brought up in the morning and measured immediately is a stack measured while it is still working.
A documented flag whose function has no caller. Setting use_raptor: true and re-parsing produces no RAPTOR task at all. The function that queues it carries a docstring saying it is triggered automatically at the tail of chunking, and a search of the whole image finds its definition and no caller. Re-parsing therefore deletes a document’s chunks, writes the same ones back, and reports done in twenty seconds.
A hard timeout that cannot fire. A @timeout(3600) in the source reads like a promise that a long build cannot exceed an hour. It enforces nothing unless an environment variable is set, and it wraps an async function synchronously, so it would time how long a coroutine takes to construct. Good news here, but the same reading tells you nothing will stop the task if it hangs.
Two score files, one mode. Measuring one mode on several index configurations leaves two result files declaring the same mode name. My own report picked between them by sorted filename, and published the retired configuration’s 65.0% as the current 67.5% until the selection was made explicit.
RAGFlow benchmark FAQ
How long does RAGFlow take to index 33 SEC filings?
103 seconds for 1,797 chunks on one AMD W7900, with no generative model involved. Indexing embeds chunks and writes them to Elasticsearch. The knowledge graph is separate and costs 5.75 hours on the same corpus.
Does RAGFlow work with a local model instead of OpenAI?
Yes, through the OpenAI-API-Compatible provider. Register two model instances, one for chat and one for embeddings, and point them at your own server. Plain HTTP is easiest; a self-signed HTTPS certificate needs a relay because no container hostname will match it.
What is the difference between RAGFlow’s hybrid and kg retrieval modes?
On this corpus, 5.7 seconds. Both returned identical context in all 40 questions and produced the same answers. Graph rows live in the same index as chunks and are ranked with them, so they reach the prompt whether or not the graph mode is requested.
Why does RAGFlow miss short documents?
The retrieval budget is six chunks per query over the whole corpus, and a long filing can win several of the six. Six chunks reached a median of four distinct documents, and in 28 of 40 questions one filing took half the prompt or more. Fourteen of these 33 filings hold three chunks or fewer.
Is RAGFlow’s knowledge graph worth the build time?
On this benchmark it moved correct answers from 9 to 10 out of 40 for 5.75 hours of GPU time, and answer-or-refuse accuracy from 67.5% to 75.0%. The gain is real and small, and it arrives whether or not you enable the graph mode.
Does a RAPTOR build reproduce if you run it again?
Not chunk for chunk. Rebuilding the same 33 files with the same settings on a clean stack gave 793 summaries instead of 797 and changed the six chunks shown to the model in 37 of 40 questions. The aggregate is steadier than the path: recall 67.5% then 72.5%, correct answers 13 then 11, and refusal accuracy identical at 85.0%. The plain index, which no model touches, reproduces exactly. Treat any structure built by a language model as reproducible in character, not in count.
Can you compare “with graph” and “without graph” in one RAGFlow dataset?
No. Building the graph changed the default mode too: 17 of 40 questions retrieved a different set of documents afterwards. A system that has had a graph built is no longer the system whose baseline you measured. Use a separate stack for each configuration.
Summary: what the RAGFlow benchmark measured
In summary, RAGFlow indexes fast, answers fast, invents nothing, and refuses sensibly. It also retrieves no better than the benchmark’s crude floor, and it fails one entire dispersion pattern in every configuration because a short filing cannot win a slot against a long one.
How to read a RAGFlow benchmark score of 57.9
The benchmark’s scorecard gives RAGFlow’s best configuration — raptor — 57.9 of 100. That is not “58% of answers correct”, and the difference is nearly double. Retrieval carries 40 points of the rubric and answer correctness carries 20, so a system can score in the high fifties while answering 13 of 40 questions correctly. Therefore read the two numbers together: 67.5% of the evidence retrieved, 32.5% of the answers right.
The most transferable result is not a score at all. It is that a structure written into the same store as your data stops being an option and becomes a property of the corpus — and that a retrieval budget counted in chunks quietly decides how many documents your model is allowed to see.
One caveat travels with every figure above, and it was measured rather than assumed. Building RAPTOR a second time moved recall to 72.5% and correct answers to 11 of 40, having changed the model’s prompt in 37 of the 40 questions. So the scorecard row belongs to the build it came from. A framework whose index is written by a language model should be quoted with the build attached, and a two-question gap between two such systems is not yet a difference.
How RAGFlow places against the other frameworks in this benchmark is deliberately not answered here. Each episode measures one system on its own terms; the ranking, per dispersion pattern and with every other variable held equal, is the subject of the final episode.
Everything behind these numbers
The full result set ships with this article rather than as a summary you have to trust. Inside the bundle below, appendix-40-answers.html lists all 40 questions with the gold answer, what each mode replied, the judge’s verdict and its reasoning, whether the figures matched and how long each answer took. It opens in a browser; nothing has to run first.
The download bundle carries the container commands, the adapter, the shared benchmark runner, the evaluator, the chart scripts and every raw result file, plus a SHA-256 manifest so you can confirm the inputs are the ones measured here. It also carries the checks: the two gates that decide whether a graph build may be published at all, and the small scripts behind the numbers that have nowhere else to come from — parse_timing.py for what a parse really cost, concentration.py for how many documents six chunks reach, and the three that make a rebuild readable: chunk_snapshot.py, raptor_delta.py and compare_runs.py. Its folder layout matches the commands above, so they run verbatim after unzipping: nvda-rag-e05-ragflow.zip (91 files, 1.4 MB).
Not investment advice. NVIDIA filings are test material for a retrieval system, nothing more.
Companion code — the fragments above are the working parts: the --add-host line that lets a container name the Windows host, the two lines of RAGFlow’s model client whose product is a 3,600-second wall, and the scenario command that owns one generation of the stack from clean containers to a scored run. The commands in this article reproduce the result from scratch.
Free ebook
Free AI Video, Generated Locally
Run Wan 2.1 in ComfyUI on your own GPU — the scripts I use, measured times, sample clips. No cloud, no API keys.
No spam. Unsubscribe at any time.


Leave a Reply