How Things Work

Embeddings & Vector Databases

Turning text into vectors where meaning becomes geometry — cosine similarity, semantic search, and the vector databases (pgvector, Pinecone, Qdrant) that make nearest-neighbor search fast.

How It Works

An embedding maps text (or images/audio) to a vector where semantically similar things sit close together. This turns 'find related content' into a geometry problem solved by nearest-neighbor search with cosine similarity. Vector databases index millions of embeddings with approximate-nearest-neighbor algorithms (HNSW) for millisecond retrieval — the foundation of semantic search, recommendations, and RAG.

Text → Vector

An embedding model maps a piece of text (word, sentence, document) to a fixed-length vector of numbers — typically hundreds to a few thousand dimensions. The model is trained so that semantically similar inputs produce nearby vectors, capturing meaning rather than exact words.

Meaning as Geometry

In this vector space, distance encodes similarity: 'dog' and 'puppy' land close; 'dog' and 'car' land far apart. Relationships even become arithmetic — the classic king − man + woman ≈ queen. Search becomes a geometry problem instead of keyword matching.

Similarity Search

To find relevant items, embed the query and find its nearest vectors — usually by cosine similarity (the angle between vectors). The top-k nearest are the most semantically similar results. This works even when no keywords overlap, because it matches meaning, not surface text.

Vector Databases & ANN

At scale you can't compare against millions of vectors one by one, so vector databases (pgvector, Pinecone, Qdrant, Milvus, Weaviate) build approximate-nearest-neighbor indexes like HNSW. They trade a tiny bit of accuracy for massive speed, returning the closest matches in milliseconds.

Key Concepts

🧮Embedding

A dense numeric vector representing the meaning of text/image/audio, where proximity ≈ semantic similarity.

📐Cosine Similarity

The standard closeness measure — the cosine of the angle between two vectors (1 = identical meaning).

📏Dimensionality

The vector length (e.g. 384, 768, 1536). More dimensions can capture more nuance, at higher storage/compute cost.

🔎Semantic Search

Retrieval by meaning rather than keywords — matches 'return an item' to 'refund policy'.

🗄️Vector Database

Storage + ANN index (HNSW/IVF) for fast nearest-neighbor search over millions of vectors (pgvector, Pinecone, Qdrant).

⚡ANN

Approximate Nearest Neighbor — trades a little accuracy for huge speed, making large-scale similarity search feasible.

Embed, store, and search (pgvector)

tsx

1// 1) Turn text into vectors with an embedding model.
2const docs = ["return policy is 30 days", "we ship worldwide", "support hours 9-5"];
3const vectors = await embed(docs);   // e.g. 1536-dim float arrays each
4
5// 2) Store them in a vector DB (here: Postgres + pgvector).
6//    CREATE TABLE chunks (id, content text, embedding vector(1536));
7//    CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
8
9// 3) Query = embed the question, then nearest-neighbor search by cosine.
10const q = await embed(["how long do I have to return an item?"]);
11//    SELECT content, 1 - (embedding <=> $1) AS similarity
12//    FROM chunks ORDER BY embedding <=> $1 LIMIT 3;   -- <=> = cosine distance
13// → returns "return policy is 30 days" even though NO keywords matched.
14
15function cosine(a, b) {                 // similarity = 1 (same) … -1 (opposite)
16  const dot = a.reduce((s, v, i) => s + v * b[i], 0);
17  const mag = (v) => Math.hypot(...v);
18  return dot / (mag(a) * mag(b));
19}

💡

Why This Matters

Embeddings are one of the most practical, widely-used pieces of modern AI — they power semantic search, recommendations, deduplication, clustering, and the retrieval half of RAG. Understanding vectors, cosine similarity, and vector databases is essential for building AI features and a frequent topic in AI-engineering interviews.

Common Pitfalls

⚠Mixing embeddings from different models — vectors are only comparable within the same model/version.

⚠Using Euclidean distance when the model expects cosine (or forgetting to normalize) — wrong rankings.

⚠Poor chunking for documents — chunks too big lose precision, too small lose context.

⚠Ignoring approximate-NN recall/latency trade-offs and index parameters at scale.

⚠Treating embeddings as exact keyword search — they capture meaning, so they can also return plausibly-similar-but-wrong matches.

Real-World Use Cases

1Search That Understands Intent

Scenario

A help center's keyword search fails users: someone types 'I want my money back' but the article is titled 'Refund Policy', so keyword matching returns nothing useful and support tickets pile up.

Problem

Lexical (keyword) search only matches surface tokens. Users phrase things differently from the docs, so relevant content is missed whenever vocabulary doesn't overlap — a fundamental limit of keyword search.

Solution

Embed every article and store the vectors in a vector database. At query time, embed the user's phrasing and retrieve the nearest vectors by cosine similarity. 'I want my money back' now matches 'Refund Policy' because their meanings — not their words — are close.

💡

Takeaway: Embeddings power semantic search: retrieval by meaning, robust to wording. They're the right tool whenever users express the same intent in many different words.

2The Retrieval Layer Behind RAG

Scenario

A team building a RAG chatbot over company docs needs to fetch the few most relevant passages for each question so the LLM can answer accurately and cite sources.

Problem

Stuffing all documents into the prompt is impossible (context limits and cost), and keyword retrieval misses paraphrased questions, so the model gets the wrong context and hallucinates.

Solution

Chunk the docs, embed each chunk into a vector DB, and at query time embed the question and retrieve the top-k most similar chunks to inject into the prompt. The embedding/vector-search layer is what makes RAG's retrieval step both accurate and fast.

💡

Takeaway: Embeddings + a vector database are the retrieval backbone of RAG. Quality semantic retrieval is the difference between a grounded answer with citations and a confident hallucination.

Join the discussion

Comments

Loading comments...

Cookie preferences

We use essential cookies for sign-in and preferences. With your permission, we also use basic analytics to improve lessons.