Imagine you're describing the location of a person in a room.
You could say:
Move 3 steps to the right.
Move 2 steps forward.
We can represent this information using two numbers:
V = [3, 2]
This is a 2-dimensional vector.
A vector is essentially an ordered collection of numbers that represents information.
Some examples:
Vector Number of dimensions
[5] 1
[3, 2] 2
[3, 2, 4] 3
[0.5, 0.8, 0.2, 0.9] 4
The number of values in a vector determines its dimensions.
Consider this vector:
V = [3,2]
Here:
First number represents the X coordinate.
Second number represents the Y coordinate.
Together, they describe a position or direction in a 2D coordinate system.
Now imagine describing a position inside a room:
X = 3
Y = 2
Z = 4
The vector becomes:
V = [3,2,4]
We now have three dimensions.
This is similar to how a 2D vector represents a point on a flat surface, while a 3D vector can represent a point in space.
Key takeaway: A vector is a sequence of numbers. Its dimensions tell us how many numbers it contains.
Imagine you want to describe a car using three properties:
Speed
Fuel level
Temperature
You could represent a car using:
Car = [80, 60, 35]
Let's assume:
Dimension Value Meaning
Dimension 1 80 Speed (km/h)
Dimension 2 60 Fuel (%)
Dimension 3 35 Temperature (°C)
This vector has three dimensions because it contains three values.
Each dimension represents one coordinate or feature in this example.
Important: A dimension does not always represent a physical measurement. In AI embeddings, dimensions are learned numerical features, and their individual meanings are usually not directly interpretable by humans.
Now let's connect these concepts to AI.
Imagine you have three sentences:
I love programming.
I enjoy writing code.
The weather is very cold.
Humans can understand that sentences 1 and 2 have similar meanings.
But how can a computer understand that?
Computers work with numerical representations. AI models can convert text into vectors that capture aspects of its meaning.
For example, imagine an AI model converts the sentences into these simplified vectors:
Sentence 1: I love programming.
[0.90, 0.80, 0.10]
Sentence 2: I enjoy writing code.
[0.88, 0.82, 0.12]
Sentence 3: The weather is very cold.
[0.10, 0.20, 0.95]
The first two vectors are more similar.
These numbers are invented for teaching purposes. Real embedding dimensions are learned by a model and cannot generally be interpreted as individual concepts.
This is the foundation of embeddings.
An embedding is a numerical vector representation of data, such as text, that captures useful relationships or characteristics learned by a model.
Our examples used only 2, 3, or 4 dimensions to make the concept easy.
Real embedding models can generate vectors with hundreds or thousands of dimensions.
For example:
Vector size Dimensions
[0.2, 0.5, 0.9] 3
Hypothetical embedding 384
Hypothetical embedding 768
Hypothetical embedding 1536
Hypothetical embedding 3072
Why so many?
Because representing the relationships and characteristics of language is a complex task. A larger vector can provide more representational capacity, although more dimensions do not automatically mean better embeddings.
Think of it this way:
3 dimensions: a very simple representation.
384 dimensions: a much richer numerical representation.
1536 dimensions: potentially more representational capacity, depending on the model and its training.
The model determines the embedding dimensions. We don't simply add extra numbers ourselves.