Search and read Sources with ripgrep

FoundationsContent SearchFirst evaluation

Glob gave you a list of possible Sources. Ripgrep, usually called through the rg command, opens those files and finds matching text.

This search-then-read pattern is one of the most useful ideas in retrieval. Search tells you where a possible answer lives. Reading a small window around the match gives you enough context to decide whether it is Evidence.

Why agents do not read the whole file

A search match is often one line. One line may be too little to understand. The full file may be hundreds or thousands of lines, which is too much to put into model context every time.

Agent Harnesses split the work into two tools:

1. search returns paths, line numbers, and matching lines;
2. read returns a bounded line range around a promising match.

That boundary saves context and preserves provenance. If the Answer uses the passage, you already know the Source and line range for its Citation.

Find an exact identifier

From course/, search for the fictional deployment error DEP-1042:

rg --line-number --fixed-strings "DEP-1042" source/acme-deploy

--fixed-strings tells ripgrep to treat the input literally instead of as a regular expression. This is the safer default when a Query contains an error code, function name, command flag, or other exact token.

The result contains three useful pieces:

path:line_number:matching text

Do not strip the first two away. A text match without its location is hard to inspect and impossible to cite precisely.

You can narrow the search when the Query suggests a format:

rg --line-number --glob "*.md" "restart limit" source/acme-deploy

Call ripgrep from Python

The Course wrapper asks ripgrep for JSON output, then turns each result into a GrepMatch:

uv run python - <<'PY'
from pathlib import Path
from buildrag.search import grep_sources, read_source_window

root = Path("source/acme-deploy")

for match in grep_sources(root, "DEP-1042", literal=True):
    print(f"\n{match.source_path}:{match.line_number}")
    print(read_source_window(
        root,
        match.source_path,
        match.line_number,
        context=2,
    ))
PY

Read the output rather than stopping when the command succeeds. You should find docs/troubleshooting.md and nearby text explaining that the restart limit was exceeded.

That nearby text is the first real Evidence you have retrieved in this Course.

Exact search is a baseline, not a toy

The Evaluation Set includes the Query What does error DEP-1042 mean? The expected Source is docs/troubleshooting.md, and the expected Evidence contains restart limit exceeded.

Ripgrep handles this case extremely well. An embedding model would add cost and may rank the exact identifier less reliably.

Try a harder Query:

rg --line-number --ignore-case "undo a bad release" source/acme-deploy/docs

You may get no useful match even though the rollback guide answers the question. The guide uses different words. This is the limitation that will eventually motivate semantic search.

Do not jump there yet. We will first make lexical search ranked and keep exact identifiers strong.

What can go wrong

Keep the search root inside the Source Collection. Limit result counts and bytes. Treat regular expressions from users as untrusted input, and do not return an entire large file when five nearby lines answer the Query.

These are not minor implementation details. They are part of the retrieval contract an Agent Harness must enforce.

You have now used the same discover, search, and bounded-read loop that coding agents use on real repositories. In the next lesson, I will show you how those tools fit around a model.

See how coding agents retrieve context