On this page
Read
The useful mental model
Think of RAG as an open-book step at answer time. A retriever searches an external knowledge source for material related to the question. The selected passages become temporary context for the language model, which uses them while generating the response. The documents are not baked into the model, and weak retrieval can still lead to a weak answer.
In plain English
The AI gets an open-book step. It looks up useful notes, puts the best ones beside the question, and then writes the answer.
Where builders use it
- 01Internal knowledge assistants
- 02Policy and documentation Q&A
- 03Customer support answers grounded in approved material
Why it matters
RAG can make answers more grounded and inspectable because the knowledge source can be updated independently and the retrieved passages can be shown. Reliability still depends on source quality, retrieval quality, and generation.
A common misconception
“RAG teaches the model the retrieved documents.”
RAG normally supplies retrieved material as temporary context during a request. It does not update the model's trained parameters in the way fine-tuning does.
“Adding RAG makes every answer correct.”
RAG can ground an answer only in what it successfully retrieves. Poor sources, missing passages, weak ranking, or unsupported generation can still produce a bad answer.
See
See the mental model
The visual is generated from the same structured knowledge as the written explanation.
A question and an external knowledge source feed a retriever. The retriever selects relevant evidence, which is placed in the language model's context to produce a grounded answer with a source.
Watch
Watch the explainer
A short visual explanation built from the same reviewed knowledge object.
RAG: find evidence, then answer
See how retrieval finds and ranks approved workshop material, places selected evidence into temporary context, and helps a language model produce a sourced answer.
Read transcript
A learner asks a workshop assistant what changed in the setup. A language model only knows what it learned in training. It may guess. RAG adds a lookup first. The question becomes a search. A retriever, the part that searches, scans approved guides and setup notes. It ranks the passages and picks the best ones. Those passages go into the model's context, the text it reads for this request. Now it can answer from them and point back to the source. The documents do not train the model. The answer is grounded and checkable, but it is not a truth machine. If the source is wrong, or the search misses the right passage, the answer can still fail. RAG changes what the model can read before it speaks.
Builder example
Answer from the latest workshop handbook
Imagine an AI Garage workshop assistant that answers participant questions from approved facilitator guides, workbook templates, and setup notes.
Failure
A language model answering from its built-in knowledge may give a generic response, miss a recent workshop change, or confidently fill a gap that the handbook does not cover.
Intervention
Index the approved workshop documents. For each question, retrieve the most relevant passages and place them, with their source titles, into the model's context before asking it to answer.
Observable result
The participant receives an answer tied to the current workshop material and can inspect the cited source. When retrieval finds no relevant passage, the assistant can say that the handbook does not provide an answer.
Sources
Go deeper
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Google Cloud - Vertex AI APIs for building search and RAG experiences
- Google Cloud - Check grounding with RAG
Last researched 10 October 2026