Build a read-only retrieval agent¶
Until now, I have chosen the retrieval order in application code. The code runs word-based search, vector search, combines the results, selects Evidence, and asks the model for an Answer in the same order every time. This is a fixed pipeline.
Now I will let the model choose which retrieval tool to call next based on what it has already found. That tool-choosing loop is what I mean by agentic retrieval.
What makes this agentic¶
If the Query contains DEP-1042, the model may choose grep first. If the user asks about plan limits, it may call structured lookup. If the wording is vague, it may try semantic search, inspect one Source, and search again with a term it learned there.
flowchart LR
Q[Query] --> M[Model chooses a tool]
M --> H[Harness validates and runs it]
H --> O[Tool result]
O --> M
M -->|Enough Evidence| A[Answer or abstain]
M -->|Need more| HThe loop is agentic because the next step is not predetermined. The tools and safety rules are still ordinary application code.
So the first design task is choosing what decisions the model may make. I start with a small set of read-only tools whose jobs are easy to tell apart.
Expose narrow tools¶
Start with five read-only capabilities:
find_sources(pattern)
grep_sources(pattern, glob, literal)
semantic_search(query, limit)
structured_lookup(name, filters)
read_source(path, start_line, end_line)
Keep them separate. One vague search() tool hides whether the model chose filenames, exact text, vectors, or records. Separate tools make the trace understandable and let you evaluate tool selection.
The descriptions should explain when each tool is useful. The model sees those descriptions when it decides what to call.
Describing a tool does not give the model permission to run any arguments it wants. The harness still decides whether each proposed call is safe and valid.
The harness, not the model, has authority¶
The model returns a proposed tool name and arguments. Your Agent Harness must validate them before execution.
For read_source, check that the path stays inside the Source Collection, the line range is positive, and the byte limit is reasonable. For structured_lookup, allow known lookup names and filter fields. Cap every result list and the number of loop turns.
Do not give the first agent a shell, file writes, unrestricted SQL, or a generic HTTP client. None of those capabilities is needed to answer questions about Acme Deploy.
The final Answer cannot tell us whether the model chose tools sensibly. To understand the agent, we also need to inspect the sequence of calls that produced it.
Evaluate the trace¶
The final Answer is only one output. Save the tool trace and ask:
- Did the first tool fit the Query?
- Did the agent retrieve an expected Source?
- Did it read enough context to support the Answer?
- Did it repeat an unhelpful call?
- Did it stop when it had enough Evidence?
- Did it abstain on an unanswerable Query?
A correct Answer reached through a lucky or unsafe trace is still a warning.
The earlier chapters already showed the bodies for file discovery, text search, bounded reading, structured lookup, and ranked retrieval. The agent adds a control loop around those same methods.
Next, we will read a real coding harness and map its find, grep, and read tools back to the concepts you just built.