Build RAG from scratch¶
“From scratch” does not mean writing your own database driver or neural network. It means keeping the decisions that make RAG work where you can see them.
By the end of this Course, you should be able to answer questions such as:
- Which Sources were eligible for this Query?
- Why did this Chunk rank above another one?
- What Evidence did the model receive?
- Where did each Citation come from?
- Did the last change improve retrieval or only change the prose?
A framework can be useful later, but it should not be the only place where those answers live.
One project, built in layers¶
The running example is an assistant for Acme Deploy, the pretend deployment platform introduced earlier. The Source Collection stays the same while each chapter adds another retrieval capability around it.
flowchart TD
F[Discover, grep, and read] --> I[Ingest and chunk]
I --> L[Ranked lexical search]
L --> S[Structured lookup]
S --> V[Exact vector search]
V --> P[PostgreSQL and pgvector]
P --> H[Hybrid retrieval and reranking]
H --> E[Evidence and Citations]
E --> G[Generate and evaluate]
G --> A[Agentic and production operation]Each layer stays in the project. We do not replace lexical search when vectors arrive, and we do not stop tracking Source locations when generation begins.
Let each failure teach the next concept¶
The sequence is driven by observable limits:
- Glob discovers files but cannot search their contents.
- Grep finds exact text but does not rank a large result set.
- Ranked word search puts likely matches first but can miss different wording with the same meaning.
- Structured lookup answers table questions without similarity.
- Exact vector search handles meaning but does not scale forever.
- pgvector persists and indexes vectors.
- Hybrid retrieval combines methods that recover different Evidence.
- Reranking helps when useful Chunks are present but ordered poorly.
- Generation turns selected Evidence into an Answer or abstains.
This is why the order matters. You should feel the limitation before I introduce the next component. Otherwise, RAG becomes a pile of tools without a reason for any of them.
Keep one Evaluation Set beside the project¶
The Evaluation Set contains exact, meaning-based, structured, multi-source, and unanswerable Queries. We will compare the same questions against each retrieval version.
When a method improves semantic recall but hurts exact identifiers, you will see both. When a prompt sounds better but Citations get worse, you will see that too.
Where you are now¶
You already understand the first retrieval loop: discover paths with glob, search content with ripgrep, and read a bounded section. The next phase turns those loose files into stable Sources and Chunks that every later component can share.