RAG is more than a vector database¶
Whenever I see people teach RAG, they usually start with a vector database.
They take a document, turn it into embeddings, store those embeddings in a vector database, and search for the closest match. After seeing the same diagram a few times, it is easy to think that RAG and vector search are the same thing.
I thought that way when I first learned RAG too.
But a vector database is only one way to retrieve information. You can retrieve information with a normal text search, a SQL query, an API call, or even by looking for a filename. If you find useful information and give it to a model before the model answers, you are already doing RAG.
Let me explain what I mean.
What RAG means¶
RAG stands for Retrieval-Augmented Generation. The name sounds more complicated than the idea.
Retrieval means finding information that can help answer a question.
Augmented means adding that information to what the model can read for this request.
Generation means asking the model to write an answer using that information.
Here is a small example.
You ask:
What does error DEP-1042 mean?
The application searches your troubleshooting guide and finds:
DEP-1042 means the deployment restart limit was exceeded.
It then sends both the question and the matching sentence to the language model. The model writes the answer from that sentence.
That is RAG. We retrieved information, added it to the model's input, and generated an answer.
There is no vector database in this example.
Where the idea came from¶
The original 2020 RAG paper described RAG as models that combine learned memory with an external memory they can search:
“We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) — models which combine pre-trained parametric and non-parametric memory for language generation.”
- Patrick Lewis and others, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
The paper uses academic language, so I will translate it.
“Fine-tuning” means training an existing model further with additional examples. “Parametric memory” means information the model learned during training. “Non-parametric memory” means information stored outside the model that it can retrieve when needed.
The paper used a vector index to search Wikipedia. That was its retrieval method. The larger idea is that the model does not have to rely only on what it learned during training. It can look up information before answering.
Where vector databases fit¶
Most RAG tutorials use a vector database because it can find text with similar meaning, even when the words are different. It does this with embeddings, which are numerical representations of text.
You do not need to understand embeddings yet. We have a full lesson later where I will show you how embeddings and similarity search work, followed by another lesson where you build vector search yourself.
For now, remember one thing: a vector database is one retrieval tool. Exact text search, SQL, and APIs can retrieve information too. We will learn each method when we reach the problem it solves.
The full RAG flow¶
I think about RAG as five small steps:
flowchart LR
Q[Question] --> R[Find useful information]
R --> E[Choose supporting information]
E --> M[Give it to the model]
M --> A[Write or refuse the answer]
A --> C[Show the source]First, the user asks a question. The application searches for useful information. It chooses the parts that support an answer and gives them to the model. The model answers from those parts, or says it does not have enough information. The application then shows where the supporting information came from.
Throughout this Course, I call supporting information Evidence. I call the link back to the original file, record, or line a Citation.
Those two words will appear often, so remember them like this:
- Evidence supports the answer.
- A Citation shows you where the Evidence came from.
Why I start with simple search¶
If I begin this Course with embeddings and a vector database, you may learn how to copy one RAG architecture without understanding why each part exists.
I would rather start with the smallest version that works.
You will first find files by name. Then you will search inside those files for exact text. Once you understand what those methods can and cannot find, embeddings will make much more sense.
The next lesson sets up the project and the files we will search. After that, we will start with glob and grep, two search tools that coding agents use all the time.