Understand embeddings and cosine similarity¶
Lexical search could not reliably connect undo a bad release with a guide that talks about a rollback. The words differ, but the meaning is close. Embeddings give us a way to compare that meaning.
An embedding model turns text into a fixed-length list of numbers. That list is called a vector. You do not read the numbers one by one. You compare one vector with another to estimate whether the two pieces of text have similar meaning.
See the comparison before the model¶
Cosine similarity is small enough to write in plain Python:
from math import sqrt
def cosine_similarity(left: list[float], right: list[float]) -> float:
if len(left) != len(right) or not left:
raise ValueError("vectors must have the same non-zero length")
left_length = sqrt(sum(value * value for value in left))
right_length = sqrt(sum(value * value for value in right))
if left_length == 0 or right_length == 0:
raise ValueError("cosine similarity is undefined for a zero vector")
dot_product = sum(a * b for a, b in zip(left, right, strict=True))
return dot_product / (left_length * right_length)
same_direction = cosine_similarity([1, 0], [1, 0]) # 1.0
right_angle = cosine_similarity([1, 0], [0, 1]) # 0.0
Imagine each vector as an arrow. Cosine similarity measures the angle between two arrows. You will get 1.0 when they point in the same direction and 0.0 when they form a right angle. The third value lands between them.
We use that score to rank Chunks by similarity. The embedding model supplies the vectors, but it does not search, filter, or cite anything for you.
You have seen how the comparison works with tiny vectors we invented. Next, let a real embedding model create the vectors from text.
Embed two sentences locally¶
A small model from the sentence-transformers Python package can create embeddings without an API key:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
texts = [
"undo a bad release",
"roll back to an earlier healthy deployment",
"compare plan build-minute limits",
]
vectors = model.encode(texts, normalize_embeddings=True)
for text, vector in zip(texts, vectors, strict=True):
print(text, vector.shape)
Compare the first vector with the other two using cosine_similarity. The rollback sentence should be closer than the pricing sentence.
Record the exact model version when you turn this experiment into an evaluation run. Also record whether the model added a prefix before the text, cut off long inputs, or normalized the vectors to the same length. Those details change retrieval results.
Before we turn this into a search system, I want to set one boundary. Similar meaning is useful, but it does not make every other retrieval method obsolete.
Know what embeddings will not fix¶
Meaning-based similarity is useful for paraphrases, which are different ways of saying the same thing. It is a poor reason to abandon exact methods. DEP-1042 should still go to word search, and the Pro build-minute limit should still come from structured lookup.
We are adding one retrieval method to the system, not replacing the methods that already work.