Build brute-force vector search¶
In the last lesson, you embedded two sentences and saw that nearby vectors can represent similar meaning. Now we will turn that comparison into a retriever.
I am deliberately starting with the slow version. For every Query, we compare its vector with every Chunk vector. The algorithm is obvious, which makes it a good reference when we add a database index later.
Rank every Chunk¶
Using the cosine_similarity() function from the previous lesson:
scores = [
(cosine_similarity(query_vector, chunk_vector), chunk_id)
for chunk_id, chunk_vector in vectors.items()
]
scores.sort(reverse=True)
for score, chunk_id in scores[:5]:
print(f"{score:.3f} {chunk_id}")
There are only two moving parts: the cosine_similarity() function from the previous lesson and the descending sort. The function rejects a zero-length vector rather than quietly producing a meaningless score.
Use the meaning-based Evaluation Query How can I undo a bad release? and inspect whether Chunks from docs/rollbacks.md move near the top. Then try DEP-1042. Vector search may understand the first Query better while word-based search remains stronger for the exact error code.
Once the ranking looks reasonable, save enough information to run the same comparison again. Otherwise, a future model or setting change can look like an unexplained improvement.
Save enough detail to reproduce the run¶
Record the embedding model and version, whether vectors were normalized, the number of values in each vector, and the elapsed time. Keep the returned Chunk IDs and scores too. Without those details, a later "quality improvement" is hard to distinguish from a model change.
You also need to choose top-K, which means the first K ranked results you return. A larger K may find the expected Evidence more often, but it can fill the Evidence budget with repetitive Chunks. Test several values against the Evaluation Set and inspect the misses.
We are about to replace this slow loop with a database. Keep the slow version, because it gives you an exact result to compare the faster system against.
Keep this baseline¶
Brute-force search becomes expensive as the Source Collection grows, but do not delete it when you add pgvector. Run a sample of Queries through both systems. If the approximate index fails to return a Chunk that exact search found, you can measure the recall loss directly.
The retrieval behavior is now correct and visible. Next, we will persist the same vectors and metadata in PostgreSQL, then decide whether an approximate index is worth adding.