Meet the example Sources¶
I use the same example throughout this Course so you can focus on how retrieval changes instead of learning a new scenario in every chapter.
The example is a pretend software product called Acme Deploy. It is not a real company or service. Imagine a platform that helps developers deploy applications. It has product guides, configuration files, plan details, source code, and deployment logs.
Those files give us realistic questions to answer:
What does error DEP-1042 mean?
How do I undo a bad release?
How many build minutes come with the Pro plan?
What I mean by a Source¶
I call any original file or record a Source. A troubleshooting guide is a Source. A CSV file containing plan details is another Source. A deployment log is also a Source.
The complete group is the Source Collection, which simply means all the information our RAG system is allowed to search.
Here is the shape of the example collection:
acme-deploy/
├── docs/
│ ├── quickstart.md
│ ├── rollbacks.md
│ └── troubleshooting.md
├── data/
│ ├── plans.csv
│ └── incidents.csv
├── config/
├── logs/
└── src/
You do not need these exact files to understand the code. The snippets use short examples from them so that paths, matches, records, and Citations stay consistent from one lesson to the next.
How I check whether retrieval improves¶
I also keep a set of 40 questions with the Sources I expect each question to find. I call this the Evaluation Set.
The questions cover five situations:
- exact words and identifiers;
- similar meaning written with different words;
- values stored in tables;
- answers that need more than one Source;
- questions the available Sources cannot answer.
When we change a retrieval method, we can ask the same questions again. That tells us whether the new method found better information or made an older case worse.
Now that the example and terminology are clear, we can begin with the smallest retrieval tool: finding files by name with glob.