Discover Sources with glob

FoundationsSource DiscoveryNo model

Whenever I use a coding agent, one of the first things I see it do is look for files. It might search for every Markdown file, every Python test, or every filename containing config. Glob is the pattern language it uses to do that.

A coding agent is a language model connected to tools that can inspect a software project. The model cannot see your whole project by default. It can only read the text its tools bring back, and there is a limit to how much text it can read at once.

Glob gives the agent a cheap way to ask, “What files could matter here?” before it spends time searching or reading them.

What glob does

A glob pattern matches paths. It looks at directory names, filenames, and extensions. It does not open the files or understand their contents.

You have probably used a glob without calling it one:

*.md

The * means “any characters within this directory level.” Here are the patterns you will use most often:

Pattern	What it matches
*.md	Markdown files in the current directory
docs/*.md	Markdown files directly inside docs/
docs/**/*.md	Markdown files anywhere below docs/
data/*.{csv,json}	CSV or JSON files in data/ when the tool supports brace expansion
**/test_*.py	Python test files at any depth

The difference between * and ** matters. One star stays at one directory level. Two stars can cross directories.

You now know how glob describes a set of paths. Before using it in an agent, compare it with the other search command you installed: grep.

Glob and grep solve different jobs

I usually pair glob with grep because they search two different things. You need to understand the difference before we use them together.

Glob searches file paths. It answers questions such as:

Which Markdown files exist under docs/?
Which Python files have names beginning with test_?

Grep opens files and searches their text. It answers questions such as:

Which file contains DEP-1042?
Which lines mention the restart limit?

In this Course, we use ripgrep, a fast grep implementation whose terminal command is rg. A basic content search looks like this:

rg "DEP-1042" acme-deploy

You will learn ripgrep properly in the next lesson. For now, keep this distinction in your head:

Tool	Searches	Typical output
glob	paths and filenames	candidate Source paths
grep or ripgrep	text inside files	matching lines and locations

The order is not fixed. An agent may grep the whole Source Collection when the user's question contains a rare error code. It may glob first when a path or file type can narrow the search. Either way, the agent needs to know whether it is searching for files or searching inside files.

Why an agent often starts with glob

Imagine you ask a coding agent, “Where is retry behavior implemented?” It could search the entire repository for retry. It may instead discover likely Python files with src/**/*.py, or locate tests with tests/test_*.py.

That first step gives the agent a map. It can search a smaller set of files, read a few relevant sections, and repeat if needed.

flowchart LR
    Q[Question] --> G[Glob for candidate paths]
    G --> S[Search file contents]
    S --> R[Read a small section]
    R --> E[Use as Evidence]

This is retrieval, even though there is no vector database. The agent has an information need, chooses a retrieval method, and brings the result into its context.

Let us leave the hypothetical agent for a moment and look at the smallest Python version.

Discover Markdown files

pathlib includes glob support:

from pathlib import Path

root = Path("acme-deploy")

for path in sorted(root.glob("docs/**/*.md")):
    print(path.relative_to(root))

For the example collection, this returns paths such as:

docs/quickstart.md
docs/rollbacks.md
docs/troubleshooting.md

Nothing inside those files has been read. We discovered candidates based only on their paths.

Path.glob() proves the idea. A reusable discovery function should also handle several patterns, ignored directories, duplicates, and stable ordering.

Build a reusable discovery function

Here is the complete function:

from collections.abc import Iterable
from pathlib import Path, PurePosixPath

IGNORED_PARTS = {".git", ".venv", "__pycache__", "node_modules"}


def discover_sources(
    root: Path,
    patterns: Iterable[str],
) -> list[PurePosixPath]:
    matches: set[PurePosixPath] = set()

    for pattern in patterns:
        for path in root.glob(pattern):
            relative = path.relative_to(root)
            if path.is_file() and not IGNORED_PARTS.intersection(relative.parts):
                matches.add(PurePosixPath(relative.as_posix()))

    return sorted(matches, key=PurePosixPath.as_posix)

The function ignores common generated directories, removes duplicate matches, and returns relative paths with forward slashes in a stable order.

Stable relative paths become important later. We will use each path to create a stable label for the file, which I call a Source ID. We will also use the paths to compare test runs and build Citations that work on more than one machine.

Where glob stops

Suppose the user asks, “What does DEP-1042 mean?” Glob may discover docs/troubleshooting.md, but only because the filename looks promising. It cannot know whether the error code is inside.

That is where the next lesson picks up. You will use ripgrep to search Source contents, keep each match's path and line number, and read only the nearby lines needed as Evidence.

Search and read Sources with ripgrep