Retrieval-Augmented Generation

What it is

Retrieval-augmented generation searches a document set for relevant passages and gives them to a language model as the basis for its answer. The term was introduced by Lewis and colleagues in 2020, though what that paper described is narrower than what now carries the name: a fine-tuning recipe against a dense vector index of Wikipedia in which, in the paper's phrase, both the generator and retriever are jointly learned. Even that is narrower than it sounds — the paper keeps the document encoder and the index fixed and fine-tunes only the query encoder and the generator. Systems built under the name today usually leave the language model itself untrained — they index a corpus, retrieve, and put the passages in the prompt at question time; where teams do train, a common target is the retrieval side, such as fine-tuning the embedding model on a domain corpus. What survived from the paper is the property it demonstrated: the index can be swapped to update what the system knows without retraining the model, which is why retrieval became the standard way to answer questions about a company's own documents.

Why it matters

RAG relocates the accuracy problem rather than solving it. The model is now only as reliable as the passages retrieved, and those are only as reliable as the corpus they came from — so an unreconciled document set produces an unreliable system no matter how good the retrieval or the model. Teams routinely tune chunking and rerankers for months while the underlying corpus still contains three different answers to the same question. Knowledge Company works on the corpus, because it sets the ceiling the other two cannot pass.