Discover Sources with glob¶
Glob is the bread and butter of an Agent Harness. It gives an agent a cheap way to ask, “What files could matter here?” before it spends time searching or reading them.
A model does not see your whole repository or document collection by default. Its context window contains only the text the application sends. If you hand it every file, you waste context and eventually run out of room. If you hand it nothing, it has to guess. Glob is one of the first tools that closes that gap.
What glob does¶
A glob pattern matches paths. It looks at directory names, filenames, and extensions. It does not open the files or understand their contents.
You have probably used a glob without calling it one:
*.md
The * means “any characters within this directory level.” Here are the patterns you will use most often:
| Pattern | What it matches |
|---|---|
*.md |
Markdown files in the current directory |
docs/*.md |
Markdown files directly inside docs/ |
docs/**/*.md |
Markdown files anywhere below docs/ |
data/*.{csv,json} |
CSV or JSON files in data/ when the tool supports brace expansion |
**/test_*.py |
Python test files at any depth |
The difference between * and ** matters. One star stays at one directory level. Two stars can cross directories.
Glob and grep solve different jobs¶
Agent Harnesses commonly pair glob with grep. You need to understand the difference before we use them together.
Glob searches file paths. It answers questions such as:
Which Markdown files exist under docs/?
Which Python files have names beginning with test_?
Grep opens files and searches their text. It answers questions such as:
Which file contains DEP-1042?
Which lines mention the restart limit?
In this Course, we use ripgrep, a fast grep implementation whose terminal command is rg. A basic content search looks like this:
rg "DEP-1042" source/acme-deploy
You will learn ripgrep properly in the next lesson. For now, keep this distinction in your head:
| Tool | Searches | Typical output |
|---|---|---|
| glob | paths and filenames | candidate Source paths |
| grep or ripgrep | text inside files | matching lines and locations |
The order is not fixed. An agent may grep the whole Source Collection when the Query contains a rare error code. It may glob first when a path or file type can narrow the search. Either way, the agent needs to know whether it is searching for files or searching inside files.
Why an agent often starts with glob¶
Imagine you ask a coding agent, “Where is retry behavior implemented?” It could search the entire repository for retry. It may instead discover likely Python files with src/**/*.py, or locate tests with tests/test_*.py.
That first step gives the agent a map. It can search a smaller set of files, read a few relevant sections, and repeat if needed.
flowchart LR
Q[Question] --> G[Glob for candidate paths]
G --> S[Search file contents]
S --> R[Read a small section]
R --> E[Use as Evidence]This is retrieval, even though there is no vector database. The agent has an information need, chooses a retrieval method, and brings the result into its context.
Try it on the Course Project¶
Change into course/, then ask Python for every Markdown guide in the Acme Deploy Source Collection:
uv run python - <<'PY'
from pathlib import Path
root = Path("source/acme-deploy")
for path in sorted(root.glob("docs/**/*.md")):
print(path.relative_to(root))
PY
You should see paths such as:
docs/quickstart.md
docs/rollbacks.md
docs/troubleshooting.md
Nothing inside those files has been read yet. You have discovered candidates based only on their paths.
Use the Course wrapper¶
Real tools need consistent ordering and ignore rules. BuildRAG wraps pathlib in discover_sources():
uv run python - <<'PY'
from pathlib import Path
from buildrag.discovery import discover_sources
root = Path("source/acme-deploy")
patterns = ["docs/**/*.md", "config/*.yaml", "data/*.json"]
for path in discover_sources(root, patterns):
print(path)
PY
Open course/src/buildrag/discovery.py after you run it. The function ignores directories such as .git, .venv, and __pycache__, removes duplicates, and returns relative POSIX paths in a stable order.
Stable relative paths become important later. We will use them to create Source IDs, compare evaluation runs, and build Citations that work on more than one machine.
Where glob stops¶
Suppose the user asks, “What does DEP-1042 mean?” Glob may discover docs/troubleshooting.md, but only because the filename looks promising. It cannot know whether the error code is inside.
That is where the next lesson picks up. You will use ripgrep to search Source contents, keep each match's path and line number, and read only the nearby lines needed as Evidence.