Text embeddings convert text into dense vector representations that capture semantic meaning. This guide covers generating embeddings with OpenAI and local models, measuring similarity, and practical applications.
A text embedding is a dense vector — typically 768 to 3072 floating-point numbers — where the values capture the semantic meaning of the text. Texts with similar meaning produce vectors that are close together in this high-dimensional space. This is why embeddings power search, clustering, and similarity detection.
Embeddings are not interpretable — you cannot look at a single number and understand what it means. They only make sense in relation to other embeddings. The distance between two embedding vectors tells you how semantically similar the corresponding texts are.
Python: `from openai import OpenAI; client = OpenAI(); response = client.embeddings.create(model='text-embedding-3-small', input='The quick brown fox jumps over the lazy dog'); vector = response.data[0].embedding`. The vector is a list of 1536 floats.
For bulk embedding, pass a list of strings in a single call: `client.embeddings.create(model='text-embedding-3-small', input=['sentence 1', 'sentence 2', ...])`. This is much more efficient than one call per text. Batch up to 2048 strings per request. For very large datasets, use the Batch API for async processing at 50% lower cost.
For privacy or cost reasons, run embedding models locally: `pip install sentence-transformers`. Then: `from sentence_transformers import SentenceTransformer; model = SentenceTransformer('BAAI/bge-m3'); embeddings = model.encode(['sentence 1', 'sentence 2'], batch_size=32, show_progress_bar=True)`.
BAAI/bge-m3 is the leading open-source multilingual embedding model (supporting 100+ languages) and matches OpenAI's embedding quality on most benchmarks. For English-only use, `mixedbread-ai/mxbai-embed-large-v1` is a strong alternative. Both run comfortably on CPU; a GPU speeds up bulk encoding 10–20×.
Cosine similarity is the standard metric for embedding comparison. In NumPy: `import numpy as np; def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))`. The result is between -1 (opposite) and 1 (identical). For practical search, values above 0.85 are typically highly relevant.
For batch similarity (comparing one query against many stored embeddings), use vectorised operations: `similarities = np.dot(query_vec, stored_vecs.T)`. This computes all similarities in a single matrix multiplication, which is orders of magnitude faster than computing each similarity individually.
Semantic search: embed all your documents at indexing time. At query time, embed the query and find the most similar documents by cosine similarity. This finds relevant documents even when they don't share keywords with the query.
Deduplication: embed all items and cluster by similarity. Items with cosine similarity above 0.95 are near-duplicates. Use this to clean training datasets, deduplicate support tickets, or find similar products.
Classification without training: embed your categories as text descriptions, embed the item to classify, and assign the category with the closest embedding. Zero-shot classification that requires no labelled training data.
Semantic search finds relevant results based on meaning rather than keywords. This guide builds a complete semantic search system from document ingestion to ranked retrieval, using embeddings and a vector store.
Read guideVector databases are purpose-built for storing and querying embeddings at scale. This guide compares the leading options — Pinecone, Weaviate, Qdrant, pgvector, and Chroma — and helps you choose the right one for your use case.
Embedding model choice significantly affects search quality and cost. This guide compares the leading models — OpenAI text-embedding-3, Cohere embed, BAAI/bge, and others — across quality, speed, cost, and language support.
Read guide