RAG, CAG, KAG and the search beneath them: which fits which problem in 2026

October 26, 2026

If you build anything on a language model, you have to decide how it finds what it needs to know, and the choices now come with acronyms: RAG, CAG, KAG, and under them full-text search, vector search, hybrid search, graphs, SQL and plain grep. On 23 October 2026 I read the papers behind each of them, the documentation and pricing pages of the engines and model APIs, and the benchmarks that compare them, to work out which fits which problem. What surprised me is how little the acronym decides. The search method underneath it decides more, and the oldest one, a ranking formula from 1994, is still the one every newer method is measured against.

In brief

1. BM25 is still the baseline nothing has retired. On the BEIR benchmark in 2021, the dense retriever DPR scored 47.7 per cent below it on average, and on the LIMIT test of 2025 BM25 reached 93.6 recall@100 where the best single-vector embedding reached 18.9. Where the words of the question appear in the document, lexical search is competitive, cheap and current.
2. Hybrid search beats either leg, and a reranker beats hybrid by more. In the neutral studies, fusing a lexical and a dense retriever gains two to five points over the better leg; a reranker over the top 100 candidates added 11 per cent in 2021 and up to 39 per cent in later tests on BEIR. Retrieve with two cheap legs, rerank once, is the configuration every study and every vendor converges on.
3. Every frontier model sells a window of a million tokens, and nobody has measured how much of it works. The benchmarks that measure usable length put it at a fraction of the claim on 2024 and 2025 models (GPT-4o at 8,000 of a claimed 128,000 on NoLiMa), and none of them has been run on the 2026 models.
4. CAG works at small scale, and the price arithmetic says where. The paper tested 21,000 to 85,000 tokens on an 8-billion-parameter model and won five of six settings. At a cache-read price of a tenth of base input, a cached corpus of 50,000 tokens costs the same per query as sending 5,000 retrieved tokens; at a million tokens it costs ten times as much.
5. Graphs pay for multi-hop and whole-corpus questions and cost everywhere else. Independent studies give graph methods about 27 points on multi-hop questions and a clear win on summarisation, while plain retrieval wins single-hop questions and, when no answer exists, says so 96 per cent of the time against 19 for global graph search. The win rates the graph papers report come from an LLM judge whose verdict swings more than 30 points with the order of the answers.
6. The coding agents went back to grep. Cursor turned off its embedding index in July 2026, Sourcegraph dropped embeddings in February 2024 and Claude Code’s documented tools are pattern and regex search; GitHub Copilot keeps a semantic index for the case where the agent does not know the names to search for.

Three acronyms, three different bets

RAG, CAG and KAG are not three versions of one thing: they are three different answers to the question of where a model’s knowledge sits at the moment it answers. Retrieval-augmented generation (RAG) fetches a few passages per question. Cache-augmented generation (CAG) loads the whole corpus into the model once and answers from the cache. Knowledge-augmented generation (KAG) restructures the corpus into a graph with a schema and reasons over that. Each name has a date and an origin.

• RAG comes from a 2020 paper from Facebook AI Research, UCL and NYU, accepted at NeurIPS 2020. Its retriever was dense, its corpus was Wikipedia, and retriever and generator were trained together. What practice kept is the shape, retrieve then generate, and the name. What it dropped is the training.
• CAG comes from a December 2024 paper from National Chengchi University and Academia Sinica, titled “Don’t Do RAG”, published as a short paper at the Web Conference 2025. Its idea is to preload a curated document collection, compute the model’s key-value cache once, store it, and answer each question from it, with no retrieval at query time.
• KAG is a September 2024 paper and an open-source framework from Ant Group’s knowledge graph team, under the Apache 2.0 licence. It is a specific system that layers expert, schema-constrained knowledge over an extracted graph over the raw text, indexes graph and text against each other, and plans an answer as a sequence of retrieval, sort, math and deduce steps. Blogs use “KAG” loosely for any knowledge-graph retrieval; none of the primary sources I read does, and I keep the two apart here. The general idea is graph RAG.

The search methods beneath them are older, and the timeline below puts the dates in one place.

The pipeline every method plugs into

Whatever the acronym, a production system runs the same stages, and each method changes one or two of them. The vendor documentation I read gives no single authoritative list, so this is the union of what Amazon Bedrock, Azure OpenAI On Your Data, Vertex AI RAG Engine and OpenAI’s file search name.

Full-text search and BM25

Lexical search is the right first choice whenever the words of the question are in the document, and that covers more than it sounds like. BM25 is a ranking function presented at the TREC-3 conference in 1994. For each query term a document contains, it multiplies an inverse document frequency weight (a rare term weighs more), a term-frequency factor that saturates (the parameter k1 sets how fast repeats stop counting) and a length normalisation (the parameter b, so long documents are not favoured for being long), and sums the products. Every engine I read ships k1 at 1.2 and b at 0.75; SQLite’s FTS5 hard-codes them.

The scoring is the small part. A full-text engine adds analysis (tokenisation, stemming, stop words; Elasticsearch ships 36 language analyzers), phrase and proximity queries, boolean operators, fuzzy and prefix matching, filters, facets and highlighting. What it cannot fix on its own is vocabulary mismatch: a 1987 study in Communications of the ACM found that two people choose the same word for the same thing with probability under 0.20. Synonym lists patch that by hand; nothing in the engine knows meaning.

Not everything called full-text search ranks by BM25, which matters if you are counting on it. Apache Lucene made BM25 its default in version 6.0.0 on 8 April 2016, so Elasticsearch, OpenSearch and Solr rank by it, as do Tantivy, Azure AI Search for services created after 15 July 2020, and Postgres extensions such as ParadeDB’s pg_search and Tiger Data’s pg_textsearch. PostgreSQL’s own ts_rank ranks by lexeme frequency with no inverse document frequency; Meilisearch, Typesense and Algolia rank by ordered tie-breaking rules instead.

Where BM25 wins

The evidence for BM25 is five years old and has not been overturned. The BEIR paper (NeurIPS 2021) ran ten systems zero-shot on 18 datasets. Against BM25’s average, DPR was 47.7 per cent lower, ANCE 7.4 per cent lower and TAS-B 2.8 per cent lower; only ColBERT (2.5 per cent higher) and BM25 with a cross-encoder reranker on top (11 per cent higher) beat it. Dense retrievers did worst on datasets with a large domain shift, and the authors called BM25 a robust baseline. Three later measurements point the same way:

The cost side is BM25’s strongest card: no embedding model, no GPU, no vector store, updates that are immediate (SQLite, PostgreSQL) or near real time (Elasticsearch refreshes a searched index every second), and the engine you probably already run. The coding agents are the extreme case, and I come back to them below: Claude Code’s search tool is built on ripgrep, which does no ranking at all, because the model supplies the keywords and reads the results.

Vector search

Dense retrieval is the right choice when the user describes rather than names, and its limits are now theorems rather than anecdotes. An embedding model maps a text to a fixed-length vector so that inputs with similar meaning land close together; retrieval is then nearest-neighbour search in that space. Three things decide what you get: the model fixes what “close” means, the index fixes how fast “closest” can be found, and compression fixes what it costs to keep the vectors.

The models and what they cost

The embedding market has moved to long inputs and adjustable sizes. Every current proprietary model I read accepts 8,192 tokens or more (Cohere Embed 5: 128,000; Voyage 4: 32,000), and all of them let you truncate the vector to fewer dimensions through Matryoshka representation learning, a 2022 training method that makes an embedding’s prefixes usable on their own. Gemini Embedding 2, stable since April 2026, embeds text, images, audio, video and PDF pages in one space. Prices run from $0.02 per million tokens (OpenAI’s text-embedding-3-small, Voyage 4 lite) to $0.20 (Gemini Embedding 2). On the MTEB multilingual leaderboard as read that day, the top mean scores belong to large open models: Microsoft’s harrier-oss-v1-27b at 74.27, Tencent’s KaLM-Embedding-Gemma3-12B at 72.32 and Qwen3-Embedding-8B at 70.58, with Google’s gemini-embedding-001 at 68.37. The leaderboard’s own zero-shot column (78 per cent for the top model) is a reminder that the margin on your data is unknown until you measure it.

The index at each scale

The index choice is settled in the documentation: HNSW everywhere, with the same defaults. HNSW, from a 2016 paper, is a layered proximity graph whose upper layers hold a sparse sample for long jumps; pgvector, Qdrant, Weaviate, Elasticsearch, OpenSearch, Redis, MongoDB, Azure AI Search, Chroma and DuckDB all ship it, nearly all with 16 links per node. Its cost is memory: the whole graph has to sit in RAM. When the vectors no longer fit, there are two routes. DiskANN (Microsoft, 2019) keeps compressed vectors in RAM and the graph on SSD, and served a billion points from 64 GB of RAM at under 3 milliseconds in its paper; Milvus, pgvectorscale, Redis and Cosmos DB ship a form of it. The object-storage designs (turbopuffer, Pinecone serverless, Amazon S3 Vectors, Chroma Cloud) partition the space and fetch partitions per query; turbopuffer quotes a cold query over a million documents at 874 milliseconds and a cached one at 14. For small collections, flat exact search is fine: Redis recommends it under a million vectors, and it is how you build the ground truth to measure any approximate index.

Compression is now a one-line setting. Binary quantization keeps one bit per dimension, 32 times smaller than float32, and with rescoring on the original vectors kept about 96 per cent of retrieval quality in a 2024 Hugging Face study and 97 to 99 per cent in Qdrant’s tests on 3,072-dimensional vectors; both are vendor figures. My arithmetic for ten million vectors at 1,024 dimensions: 40.96 GB in float32, 10.24 GB as int8, 1.28 GB binary, plus 1.3 to 1.6 GB for the HNSW graph whatever the vector format, which is why the graph, not the vectors, becomes the memory cost once you quantize.

Filtering divides the stores more than the index does. If most queries carry a tenant, a date or a permission filter, a store that applies the filter during the graph walk (Weaviate, Qdrant, Vespa, Elasticsearch, OpenSearch, Milvus, Azure AI Search) behaves differently from one that filters after the scan: pgvector applied filters after the index scan until version 0.8.0 added iterative scans, and Azure measured prefiltering about seven times slower at under 2 per cent selectivity on a million vectors.

Where dense retrieval fails

The failures are documented and specific. The LIMIT paper (Google DeepMind, ICLR 2026) proves that the number of distinct top-k sets a single-vector model can return is bounded by its dimension, and built a corpus of 50,000 documents and queries as simple as who likes what to show it.

Three other failures are measured. On NevIR, pairs of documents that differ only by a negation, most retrievers rank at or below random; cross-encoders do best, late-interaction models next, bi-encoders last. On EntityQuestions, dense retrievers underperform sparse methods on rare entities unless the question pattern was in training. And Collapse of Dense Retrievers (ACL 2025) found two common retrievers prefer shorter documents, early answer positions, repeated entities and literal matches; with several biases combined they picked the answer-bearing document under 10 per cent of the time, and feeding their preferred documents to a model cut accuracy 34 per cent below giving it nothing. The fix in each case is not a better embedding model but a second signal: lexical search fused with the dense results, metadata filters, late interaction, or a reranker.

Late interaction is that second signal built into the vector. ColBERT (2020) keeps one vector per token and scores by the best match per query token; ColBERTv2 cut its storage six to ten times. Qdrant, Vespa, Weaviate and LanceDB hold multi-vectors natively; Elasticsearch’s rank_vectors field is in preview and brute force only; Milvus’s “multi-vector search” means several vector fields fused, not late interaction. Budget two to three times the storage of a single vector and run it as a reranker over a first-pass candidate set, which is how the stores expect it.

Hybrid search and rerankers

Hybrid search pays when the two legs fail on different queries, and the neutral evidence says they usually do. A hybrid query runs a lexical retriever (BM25 or a learned sparse model) and a dense retriever in parallel and merges the two ranked lists. The usual merge is reciprocal rank fusion (RRF), from a two-page SIGIR 2009 paper by the University of Waterloo: each document scores the sum over the lists of 1 divided by (60 plus its rank). The constant 60 was fixed in a pilot and the authors found the choice was not critical; the pilot’s table runs from 0.2134 at k equal to 20 to 0.2147 at 80. Every engine I read ships it with 60: Elasticsearch, OpenSearch, Azure AI Search, Weaviate, Milvus and Vespa.

The sharper finding is that a tuned weighted sum beats RRF, if you have labelled queries to tune it on. An analysis of fusion functions (2022, revised 2023) ran BM25 and a small dense model on nine datasets: a convex combination with min-max normalisation beat RRF on every one (p below 0.01), both beat both legs on eight of nine, and the weight converged on under 5 per cent of a training set. Elastic’s own test on BEIR in July 2023 found the same split: RRF gave 18 per cent over BM25, a tuned linear combination 24 per cent, but the weight was model-specific. The gain from tuning is real and second-order; the gain from having two legs is first-order.

Learned sparse retrieval is the lexical leg upgraded. SPLADE (2021) uses a language model to decide which terms to add to a document and how to weight them, then stores the result in an ordinary inverted index; SPLADE++ reached 50.5 to 50.7 average nDCG@10 on 13 BEIR sets against BM25’s 43.7 and ColBERTv2’s 49.7. Elastic’s ELSER and OpenSearch’s neural sparse models are the product forms; ELSER is English only. What the evidence does not show is a sparse model replacing the dense leg on short queries: BGE-M3’s own table has its sparse head at 53.9 against its dense head at 69.2.

Rerankers move the numbers more than fusion does

A reranker is a model that reads the query and each candidate together and re-scores them, and across every table I read it moves the metric more than any fusion choice. In BEIR itself, a BERT cross-encoder over BM25’s top 100 added 11 per cent and was best on 16 of 18 datasets. Later figures, by depth:

The BRIGHT row with the cross-encoder is the caveat: a small reranker trained on MS MARCO can hurt on queries unlike MS MARCO, so the reranker’s training distribution matters as much as its size. The other cost is latency. BEIR’s own table puts BM25 at 20 milliseconds per query on a CPU and a cross-encoder over 100 candidates at 450 milliseconds on a GPU or 6.1 seconds on a CPU. Prices have converged on tokens: Cohere’s Rerank 5, released 21 October 2026, costs $0.05 per million tokens for the Pro model and $0.02 for Fast; Voyage’s rerank-3 the same two figures. Reranking 100 documents of 500 tokens is about 55,000 tokens, so roughly $0.003 a query at the higher rate, by my arithmetic. The usual depth in the sources is 100 in, 10 to 20 out.

One more lever sits on the query side. Letting the model write the query helps lexical search most: on BRIGHT, GPT-4 reasoning written into the query took BM25 from 14.5 to 27.0, more than any dense model gained; query2doc lifted BM25 by about 15 points of nDCG@10 on TREC DL 2019 and 2020, at the cost of an LLM call of over two seconds. HyDE does the same for an unsupervised dense retriever. Nothing I read re-measures these on 2026 models.

RAG as a pipeline, and where it breaks

Most of what goes wrong in a RAG system is specific and measured, and the fixes with evidence are the cheap ones. A survey of RAG (2023, revised 2024) names the stages naive, advanced and modular; what follows is what the measurements say about each part.

Chunking matters less than it is argued about. The one controlled study I found is Chroma’s evaluation of July 2024, vendor research: the choice of strategy moved recall by up to 9 points, a recursive splitter at about 200 tokens with no overlap was consistently high without winning any one metric, and the 800-token with 400 overlap default of OpenAI’s vector stores was the weakest tested. The vendors’ own defaults disagree with each other (Bedrock about 300 tokens, OpenAI 800 with 400 overlap, Vertex 1,024 with 256), which tells you they are not tuned to a shared benchmark. The fix with the larger reported gain is to give each chunk its context rather than to find the perfect boundary: Anthropic’s contextual retrieval prepends a model-written sentence or two to each chunk before indexing, and reports a 35 per cent cut in top-20 retrieval failures from that alone, 49 per cent with BM25 added and 67 per cent with a reranker, on a corpus and queries that are not public. Late chunking (2024) embeds the whole document first and pools into chunks afterwards for the same reason.

The failure points have names. Seven Failure Points When Engineering a RAG System (2024) is an experience report from three deployed systems: missing content answered anyway, the right document ranked below the top k, retrieved then dropped at consolidation, in the context but not extracted, the wrong format, the wrong specificity, and an incomplete answer when the key points span several documents. Its authors say validation is only feasible during operation, because the labelled pairs offline evaluation needs are usually not there.

More context is not more answers. Lost in the Middle (TACL 2023) showed accuracy highest when the relevant passage is first or last in the prompt and lowest in the middle. The Power of Noise (SIGIR 2024) found high-scoring passages that do not contain the answer hurt. More Documents, Same Length (2025) held the total length constant and varied only the number of documents: accuracy fell by up to 20 per cent for most models. And order-preserving RAG, below, found an inverted U as the number of retrieved chunks grows.

Retrieval raises accuracy and hallucination together. On Meta’s CRAG benchmark (NeurIPS 2024), GPT-4 Turbo alone was 33.5 per cent accurate and hallucinated 13.5 per cent of the time; with straightforward RAG it was 43.6 per cent accurate and hallucinated 30.1 per cent, because it stopped declining to answer. Evaluation has settled on a triad for this: is the retrieved context relevant, is the answer grounded in it, does it answer the question (RAGAS, ARES, TruLens), all scored by an LLM judge, which carries its own biases of position, verbosity and self-preference.

Permissions are a filter at retrieval, not a check after generation. Azure AI Search documents the pattern as a security filter applied to every query, with the group identifiers stored on each document and, if a skillset chunks documents, projected onto each chunk; Elasticsearch enforces a role query on the read APIs. This is the one requirement no full-context design meets short of building a different prompt per user, and it decides the architecture before quality is measured.

Long context and CAG

Putting the whole corpus in the prompt is a size-and-stability bet, and the sources put the size small. Every frontier API model documented on 23 October 2026 offers a window of about a million tokens: OpenAI’s GPT-6 family at 1.05 million, all four current Claude models at a million, Gemini 4 Argon at 1,048,576, DeepSeek V4 at a million, Mistral Large 4 at a million in public preview, and Llama 4 Scout at ten million. None of those pages states an effective length.

What the benchmarks say about the window

The benchmarks that measure usable length keep finding it a fraction of the claim, on the models they tested. On RULER (2024), of 17 models all claiming 32,000 tokens or more, only half kept satisfactory performance at 32,000. On NoLiMa (ICML 2025), whose needles share no words with the question, 11 of 13 models claiming 128,000 or more fell below half their short-context score at 32,000; its effective lengths were 8,000 for GPT-4o against a 128,000 claim, 4,000 for Claude 3.5 Sonnet against 200,000, and 2,000 for Gemini 1.5 Pro against two million. HELMET (ICLR 2025) found, across 59 models, that near-perfect needle-in-a-haystack scores do not predict downstream performance. Chroma’s context rot report of July 2025, a vendor report on 18 models, found performance degrading with input length in every experiment, and Anthropic’s own context windows page now uses the term. None of these has been run on the models that advertise a million tokens today, so the honest statement is that effective length is unmeasured for the current generation.

The papers that compare long context with retrieval head to head do not agree, and the good ones say why. A Google study (EMNLP 2024) found long context beat RAG on average with strong models (Gemini 1.5 Pro 49.70 against 37.33), that the two gave identical answers on 63 per cent of queries, and that routing (answer from retrieved chunks, fall back to the full context when they look insufficient) cut cost 65 per cent at comparable quality. In Defense of RAG (NVIDIA, 2024) found that keeping retrieved chunks in document order and sending 48,000 tokens beat the same model’s full 117,000-token context, 47.25 against 34.26 F1. LaRA (2025) ran 2,326 cases on 11 models and concluded there is no silver bullet. I found no 2026 study that measures the question on 2026 models.

What the CAG paper did

CAG’s own result is a small-corpus result. The paper preloads a document collection, computes the key-value cache once, answers from it and truncates the question’s tokens between queries. It tested six settings of 21,000 to 85,000 tokens from SQuAD and HotpotQA on Llama 3.1 8B, against BM25 and dense RAG built with LlamaIndex, scored by BERTScore. CAG scored highest in five of the six; on the 85,000-token HotpotQA set it fell below sparse RAG at top-3 and top-5. Its generation time was 0.85 seconds at 21,000 tokens and 2.26 at 85,000, against about 0.7 for sparse RAG at top-3 and 92 seconds for loading the same text uncached. The authors say the advantage narrows as the text grows, that the method becomes impractical for significantly larger datasets, and that a hybrid which preloads a core and retrieves for the edges is the next step. The paper has no sentence on refreshing the cache when documents change.

Prompt caching is what the APIs sell

What you can buy is prefix caching, which is CAG’s mechanism as a hosted feature, and its prices decide where it pays. Anthropic caches a byte-identical prefix for five minutes, refreshed on use, or an hour, at 1.25 or 2 times the input price to write and a tenth to read (a twentieth on Opus 5.5 and Sonnet 5.5, a fortieth on Fable 5.1). OpenAI caches automatically above 1,024 tokens, with writes at 1.25 times for a 30-minute lifetime or 2.5 times for 12 hours on GPT-5.6 and later, and reads at a tenth. Gemini adds a storage charge of $4.50 per million tokens per hour on its Pro models for explicit caches, and its newest model offers implicit caching only. DeepSeek charges nothing to write or store and about a thirtieth to read. No API lets you edit a cache: a changed document is a new prefix and a new write.

The arithmetic from the published prices, which is mine, says where the line is. Input only, no output, no question tokens:

Sending ten chunks of 800 tokens instead, 8,000 tokens, costs $0.016 on Claude Sonnet 5.5, GPT-6.1 Sol or Gemini 4 Argon, a sixty-second of the full corpus. Caching narrows that: at a read price of a tenth of base input, a cached corpus of 50,000 tokens costs the same per query as a 5,000-token retrieval call, 200,000 tokens costs twice as much, and a million ten times, twenty on OpenAI above its 272,000-token threshold where input prices double. The lower read multiples move the break-even to 100,000 tokens (Opus 5.5, GPT-6.1 Sol) or 200,000 (Fable 5.1). A write pays back on the second hit for the five-minute and 30-minute lifetimes and the third for the one-hour and 12-hour ones, but only if traffic keeps the entry alive; sparser than one query per lifetime, every query is a write and caching costs more than not caching. Gemini’s storage needs about 2.5 queries an hour to pay for itself on the lower tier, whatever the corpus size. None of this counts the embedding model, index and infrastructure that a retrieval system pays for and none of the pages prices.

The practical pattern in the CAG paper’s conclusion and in the docs’ own examples is a split: cache what is stable (the instructions, the reference manual, the many-shot examples, the tool definitions) at the front of the prompt, and retrieve or append what changes behind it. That is prompt caching used well rather than CAG as the paper defines it. The hosted notebooks are built around the same shape: Gemini Notebook, the renamed NotebookLM, caps sources per notebook at 50 to 600 by plan and each source at 500,000 words; Claude Projects holds project knowledge in context and switches to a retrieval mode near the limit.

Graph RAG and KAG

A graph earns its indexing cost on two kinds of question and costs you on the rest, and the independent evidence of 2025 and 2026 is clear about which is which. Microsoft’s GraphRAG paper (April 2024, revised February 2025) extracts an entity graph from 600-token chunks with an LLM, clusters it with the Leiden algorithm, writes a report for every community, and answers “global” questions such as what the main themes are by map-reduce over those reports. On two corpora of about one and 1.7 million tokens, an LLM judge preferred its answers to vector RAG’s 72 to 83 per cent of the time on comprehensiveness and 62 to 82 per cent on diversity, while vector RAG was more direct. Indexing the one-million-token corpus took 281 minutes with gpt-4-turbo; a global answer at the leaf level read up to 746,100 tokens of reports. The repository (MIT, version 3.3.0 on 8 October 2026) now describes itself as a research project in maintenance mode. Microsoft’s follow-ups attack the cost: dynamic community selection cut global-search tokens 77 per cent, and LazyGraphRAG, announced in November 2024 and available only through Microsoft Discovery and an Azure Local preview, claims indexing at 0.1 per cent of GraphRAG’s cost; that figure is the vendor’s and no paper was published for it.

The variants pick different trade-offs. LightRAG (EMNLP 2025) keeps the extracted graph, drops the community reports and retrieves entities and relations at two levels, with incremental updates. HippoRAG 2 (Ohio State, ICML 2025) builds an open-information-extraction graph with passages as nodes and scores them by Personalized PageRank from the entities the question names. RAPTOR (2024) is a tree of recursive summaries with no graph at all. Zep’s Graphiti adds time to every fact so that new information invalidates old edges rather than deleting them, for agent memory. KAG, described above, adds a schema layer and plans answers as logical forms; on HotpotQA, 2WikiMultiHopQA and MuSiQue it reports 76.2, 76.2 and 48.7 F1 against 63.7, 57.1 and 36.5 for IRCoT with HippoRAG on the same model, and its authors list multiple LLM calls at build and solve time, about twice the tokens of plain decomposition, and the labour of schema-constrained extraction as its costs. Its last release was 0.8.0 in June 2025.

The evidence both ways

The independent runs separate the question types. The table names who ran each.

Two things in that table changed how I read the graph papers. The first is the judge: every win rate in the GraphRAG and LightRAG papers comes from an LLM comparing pairs of long answers on comprehensiveness and diversity, on questions an LLM generated, and the corrected protocol moves those numbers by tens of points. Ask for exact match or recall on a fixed set, and for the token bill beside it. The second is that the cheapest way to get the multi-hop gain is not the community-summary pipeline but a PageRank graph over passages, which loses nothing on simple questions, or an agentic loop over plain retrieval, which pays at inference time instead of index time.

Text-to-Cypher, the free-form way to query a graph you already have, is still weak: Neo4j’s own benchmark put GPT-4o and its fine-tuned model at about 30 per cent execution match. The paths the vendors document instead are templated queries, vector-then-traverse retrievers, and a graph store that holds the embeddings beside the nodes, which Neo4j (vector index generally available since 5.13, binary quantization the default since the 2026.08 release), FalkorDB, Memgraph, TigerGraph and Amazon Neptune Analytics now all do.

Structured and agentic retrieval

When the answer is a number, a filter or a join, the right retriever is a query against the store that already governs the data, and when the corpus is live or has a fast exact-match engine, the right retriever is a tool the model drives. Neither is RAG in the usual sense, and both are what the most-used systems of 2026 do.

Text-to-SQL and the semantic layer

Text-to-SQL is close to solved on the academic benchmarks and far from solved on enterprise ones. On BIRD, the leaderboard tops out at 82.95 per cent execution accuracy against a human figure of 92.96. Spider 2.0 (ICLR 2025), built from real enterprise warehouses with databases of over a thousand columns, had o1-preview solving 21.3 per cent with an agent and GPT-4o 10.1 per cent at publication; its error analysis puts 27.6 per cent of failures in schema linking and 8.3 per cent in joins, because the warehouses often lack explicit foreign keys. Every vendor product is built the same way in response: Databricks Genie, Snowflake Cortex Analyst, BigQuery’s data agents and Microsoft Fabric’s data agents give the model a schema plus a curated layer of descriptions, synonyms, business rules and verified queries, and rank that layer above the raw schema. Snowflake’s documentation says plainly that a bare schema lacks business definitions and metric logic, and every one of those pages tells the user to check the result. The work is in the semantic layer, not the prompt.

Agentic search, and what the coding agents did

Tool-driven search has a paper trail and a product trail that now agree. ReAct (2022) interleaved reasoning with calls to a search API; Search-R1 (2025) trained 3-billion and 7-billion-parameter models to decide when to search, for 20 and 41 per cent gains over RAG baselines; a 2025 survey of agentic RAG, revised April 2026, sorts the designs into routers, multi-agent, corrective, adaptive and graph-based. The “deep research” products of OpenAI, Google, Anthropic and Perplexity all describe the same loop: dozens of searches, hundreds of sources, minutes of run time, inline citations.

The coding agents are the clearest case of a tool beating an index. Claude Code’s documentation describes its search as finding files by pattern and searching content with regex, built on ripgrep, and names no index. Sourcegraph dropped embeddings for Cody Enterprise in February 2024 and ranks with an adapted BM25 and tuned signals, citing the cost of sending code to an embedding API, of maintaining the vectors, and of searching them across more than 100,000 repositories. Cursor, which had indexed every repository with embeddings, describes its search now as a local grep engine that stores no embeddings for search, and its staff said on its forum in July 2026 that the semantic path was no longer helping in a meaningful way. GitHub Copilot is the one that kept a semantic index, for the case where the agent does not know the exact names to search for, and its documented sequence is semantic search, then grep, then usages. Aider sends a tree-sitter symbol map ranked by a graph algorithm instead of searching at all. The measurements are few: SWE-agent found that how search results are shown matters more than whether search exists (summarised search 18.0 per cent on SWE-bench Lite, no search 15.7, one result at a time 12.0), and Better Call Grep (ISSTA 2026) found an LLM issuing ripgrep commands comparable to graph-based retrieval for repository-level completion. The reasons the products give are the same three: a repository already has grep and a type checker, an index goes stale on every edit, and the file system already enforces the permissions.

Web search and agent memory

Live search replaces an index for anything that changes faster than you would rebuild it, and the price is now a line item: Anthropic’s web search tool and OpenAI’s cost $10 per 1,000 calls plus tokens, Google’s grounding $14 per 1,000 on Gemini 3 and newer after a free allowance, Perplexity’s search API $5, Brave’s $5, and all four model vendors return citations as spans with a URL and a title. Agent memory is retrieval over the agent’s own past, and its benchmarks do not yet separate the store from the agent: on LoCoMo, Mem0’s paper reports 66.88 per cent for itself and 72.90 for the full conversation in context, and Letta scored 74.0 with no memory system at all, only the conversation as a file and grep. The products agree on a shape: small notes always in context, plus a searchable archive the model reads on demand, which is the coding agents’ pattern again.

Which fits which problem

The question’s shape decides the method, and the table below is what the evidence above supports. Each row carries its caveat, because none of these is a verdict without one.

The flowchart walks the same decisions in order.

A practical order of trial follows from the same reading: hybrid search with a reranker first, because it is cheap and every neutral study puts it above either leg; an agentic loop over it when questions need more than one hop; a passage graph or a summary tree when the loop is too slow or too expensive; community summaries only for the global questions a product actually asks; and a schema-backed graph when the domain has rules and numbers that an expert will maintain. Prompt caching sits beside all of them, for the part of the prompt that does not change.

Method and caveats

• Everything above was read on 23 October 2026: the papers on arXiv and the conference pages, the engines’ and model vendors’ documentation and pricing pages, the MTEB leaderboard in a browser, and the repositories’ release pages. I installed and ran nothing, and I measured nothing myself; the only arithmetic that is mine is the memory and price tables, computed from the published figures and said to be so.
• Figures from a vendor about its own product (Anthropic’s contextual retrieval, Chroma’s chunking and context-rot reports, Elastic’s hybrid and reranker tests, Qdrant’s and Hugging Face’s quantization results, Microsoft’s GraphRAG and LazyGraphRAG win rates, Mem0’s and Letta’s memory scores) are named as such where they appear. The neutral measurements are the published papers and the independent benchmarks.
• The model lineup, context windows and prices are as the vendor pages showed them that day: Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5; GPT-6 Astra and GPT-6.1 Sol; Gemini 4 Argon and 3.1 Pro Preview; DeepSeek V4 Pro at its peak rate. Gemini’s pages announce price rises from 1 December 2026 and 1 January 2027, so the cost chart is a snapshot.
• Left out as unverified: words attributed to Claude Code’s creator in podcasts and forum posts, Amazon Bedrock’s dollar prices (the page did not render), Cohere’s and Mistral’s per-token embedding prices, Microsoft’s Azure AI Search hybrid-ranking figures, ChatGPT Projects’ file limits, Mistral Large 4’s context window, and a current BM25 row on the MTEB leaderboard.
• The long-context benchmarks measured 2024 and 2025 models; the comparison papers used Gemini 1.5 Pro, GPT-4o and Llama 3.1; the CAG paper used Llama 3.1 8B. No study I found re-runs any of them on the models that advertise a million tokens in October 2026, so the usable length of the current generation is unmeasured, and that gap is the largest one in this article.
• Papers are named by title and institution; their authors are not named here.