In our previous lessons, we learned:
Vectors: Ordered collections of numbers.
Dimensions: The number of values in a vector.
Embeddings: Numerical representations of text generated by AI models.
Now comes an important question:
If two sentences are converted into vectors, how do we determine whether their meanings are similar?
The answer is: we compare their vectors using mathematical similarity measures. One of the most commonly used methods is Cosine Similarity.
Imagine your application contains these three documents:
Document Content
A Material requirements
B Material specifications
C Employee holiday policy
A user searches:
What are the requirements for Material xyz?
Which document should the system return?
Document A might be relevant.
Document B is probably highly relevant.
Document C is unrelated.
A keyword search might miss Document B if the exact words don't match.
But if we convert the query and documents into embeddings, we can compare their numerical representations.
Cosine similarity measures the angle between two vectors.
The key idea:
Smaller angle generally means higher similarity.
Larger angle generally means lower similarity.
The direction of vectors matters, not just their length.
Consider these examples:
These are idealized geometric examples. Actual similarity scores from embedding models depend on the model and data.
The cosine similarity formula is:
Let's break it down.
A: First vector.
B: Second vector.
A.B: Dot product of the vectors.
||A|| Magnitude (length) of the first vector.
||B||: Magnitude (length) of the second vector.
Don't worry about the formula yet. Let's understand it through a numerical example.
Suppose we have two simple vectors:
A=[1,2]
B=[2,3]
This indicates that the two vectors point in very similar directions.
Let's connect everything we've learned.
Imagine your application has 10,000 documents.
A user asks:
Find documents about Material xyz specifications.
Your application performs these steps:
For example, imagine these illustrative scores:
Document Similarity
xyz specifications 0.94
xyz equipment requirements 0.82
Office stationery policy 0.12
The system ranks the first document highest.
These scores are hypothetical, not actual search results. Also, similarity is not the same as relevance: a high score doesn't guarantee that a document answers the question.
Concept Explanation
Dot product Multiplies corresponding vector values and adds them
Magnitude Measures the length of a vector
Cosine similarity Compares the direction of two vectors
Similarity score Numerical measure of directional similarity
Semantic search Searches information using meaning rather than exact keywords
A few practical cautions:
Cosine similarity is undefined for a zero vector.
Both vectors must have the same number of dimensions.
Compare embeddings generated by compatible models.
Don't assume a score of 0.8 always means 80% semantic similarity.
Similarity thresholds should be tested against real data.