In our previous lessons, we learned:
Vectors: Numerical representations containing multiple dimensions.
Embeddings: Vectors generated by AI models to represent the meaning of text.
Cosine similarity: A way to compare two vectors and measure how closely their directions align.
Now we have an important question:
Once we generate thousands or millions of embeddings, where do we store them, and how do we find the most relevant ones quickly?
That's the problem vector databases help solve.
Imagine we have 100,000 technical documents related to a material equipment.
Some documents contain:
Equipment specifications
Equipment grades
Equipment dimensions
Equipment pressure ratings
Manufacturer information
A user asks:
"Find documents related to equipment strength and pressure resistance."
A traditional database can search using exact text, keywords, filters, and indexes. But the user might search for "durability" while the document says "yield strength."
The words differ, but the meanings are related.
This is where embeddings help.
The semantic search process
1. User question
"Find information about pipe strength"
2. Embedding model
Converts the question into a numerical vector
3. Vector database
Finds document vectors with similar meanings
4. Relevant documents
Returns the most relevant document content
A vector database is a database designed to store numerical vectors and efficiently search for vectors that are similar to a given query vector.
It usually stores more than just vectors:
ID: Unique identifier for the document or chunk.
Vector: The embedding generated by the embedding model.
Text: Original text or a reference to the source document.
Metadata: Additional information such as document type, material number, asset, supplier, or date.
For example, imagine storing three document chunks:
ID Document content Embedding
101 pipe yield strength specifications [0.21, 0.85, 0.34, ...]
102 bit operating instructions [0.72, 0.13, 0.45, ...]
103 Pipe pressure and collapse resistance [0.19, 0.82, 0.37, ...]
These vectors are illustrative, not actual model-generated embeddings.
When the user searches for pipe strength, the system generates a query embedding and compares it with the stored embeddings. Documents 101 and 103 might rank highly.
Yes! This is an important distinction.
You do not necessarily need a separate vector database.
Consider two approaches:
Approach A — Store vectors in an ordinary database
Possible, but limited at scale
For example, store each vector as an array, JSON value, or a collection of numeric columns.
You could retrieve vectors and calculate cosine similarity in C#.
Problems:
You may need to load many vectors into application memory.
Similarity calculations can become expensive.
Efficient nearest-neighbor indexing is not available in every ordinary database.
Scaling the search can become difficult.
Approach B — Use a database with vector search capabilities
Designed for vector search
A vector-enabled database can use specialized indexes and search algorithms.
Benefits:
Efficient similarity search.
Search across large collections of vectors.
Metadata filtering.
Ranking and retrieving the nearest matching records.
The important point: a vector database is not simply a place to store arrays of numbers. Its main value is efficient similarity search.
Suppose you have one million document embeddings, each containing 1,536 dimensions.
A simple approach would be:
Compare the query vector with document 1.
Calculate cosine similarity.
Compare with document 2.
Repeat for every document.
Sort all similarity scores.
Return the top 5 results.
This is called exact nearest-neighbor search (often implemented as brute-force search).
For large datasets, databases often use Approximate Nearest Neighbor (ANN) search.
ANN uses specialized indexing structures to avoid comparing the query with every single vector.
For example, it may organize vectors into neighborhoods or graph structures and explore promising candidates first.
Feature Exact search ANN search
Accuracy Exact nearest neighbors Approximate
Speed at scale Can be slow Usually faster
Indexing Not necessarily required Specialized index
Best suited for Smaller datasets, exact results Large datasets, fast retrieval
One important concept: ANN introduces a trade-off between search speed and recall.
Recall tells us how many of the genuinely relevant nearest neighbors the search successfully retrieves.
You don't need to implement these algorithms yourself, but as an architect you should recognize their names.
HNSW (Hierarchical Navigable Small World): Builds a graph connecting nearby vectors. Search navigates the graph to find likely matches.
IVF (Inverted File Index): Groups vectors into clusters and searches selected clusters rather than the entire collection.
Flat index: Compares against all vectors, giving exact results rather than approximate results.
Different vector database products support different indexes and configurations.
Here are some technologies worth knowing:
Term Meaning
Vector database Stores and searches numerical embeddings
Similarity search Finds vectors similar to a query vector
Nearest neighbor A vector closest to the query
Exact search Compares against every vector
ANN Finds likely nearest neighbors efficiently
HNSW Graph-based ANN indexing algorithm
Metadata filtering Filters results using attributes such as AssetId or SupplierId
Hybrid search Combines keyword search and vector search