Embeddings

Embeddings convert text into numerical vectors that capture meaning. Similar texts produce similar vectors. This enables semantic search, recommendations, and clustering — things traditional keyword search can't do.

Concept

Text → Embedding Model → Vector

"I love pizza"    → [0.23, -0.11, 0.87, ...]
"Pizza is great"  → [0.21, -0.09, 0.85, ...]
"The stock market" → [-0.45, 0.62, 0.03, ...]

The first two vectors are close together (similar
meaning). The third is far away (different topic).

Here's how to generate embeddings using a typical API. The response is a vector of floating-point numbers.

API Request

import openai

response = openai.embeddings.create(
    model="text-embedding-3-small",
    input="How do I reset my password?"
)

vector = response.data[0].embedding
print(f"Dimensions: {len(vector)}")
print(f"First 5 values: {vector[:5]}")

The embedding is a high-dimensional vector. Common models produce vectors with 256 to 3072 dimensions.

Response

Dimensions: 1536
First 5 values: [0.0023, -0.0142, 0.0271, -0.0085, 0.0019]

Semantic search is the most common use case. You embed all your documents, store the vectors, and then find the most similar ones to a user's query.

Semantic Search Example

from numpy import dot
from numpy.linalg import norm

def cosine_similarity(a, b):
    return dot(a, b) / (norm(a) * norm(b))

# Pre-computed document embeddings
docs = [
    {"text": "How to reset your password",
     "vec": embed("How to reset your password")},
    {"text": "Billing and payment options",
     "vec": embed("Billing and payment options")},
    {"text": "Change your login credentials",
     "vec": embed("Change your login credentials")},
]

# User query
query_vec = embed("I forgot my password")

# Find most similar
results = sorted(docs,
    key=lambda d: cosine_similarity(query_vec, d["vec"]),
    reverse=True)

print(results[0]["text"])

Even though the query "I forgot my password" doesn't share exact words with "How to reset your password", the embeddings are close because the meaning is similar.

Response

Most similar: "How to reset your password"
Similarity: 0.92

Second: "Change your login credentials"
Similarity: 0.84

Third: "Billing and payment options"
Similarity: 0.31

Key takeaway: Embeddings are the foundation of semantic search and RAG systems. They let you find content by meaning, not just keywords. Use them whenever you need to match, compare, or cluster text.

Use cases for embeddings:
- Semantic search
- RAG retrieval (see RAG example)
- Duplicate detection
- Recommendation systems
- Clustering similar documents
- Anomaly detection

Vector databases: Pinecone, Weaviate, Chroma,
pgvector, Qdrant, Milvus