Vector Databases & Embeddings

By the end of this lesson you'll be able to turn data into vectors, search them by meaning with the right similarity metric, and pick between FAISS, Pinecone, Chroma, pgvector, and Weaviate.

Learn Vector Databases & Embeddings in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

A traditional database is a card catalogue: it finds a book only if you know its exact title or author. A vector database is the brilliant librarian who finds books by meaning, not by title . You say "something like Harry Potter," and they walk you to other fantasy novels with magic and boarding schools — even though none share the words "Harry" or "Potter."

How does the librarian do it? In their head, every book has a position based on its themes: fantasy books cluster in one corner, cookbooks in another. "Closeness" in that mental map means "similar in meaning." A vector database makes that map literal: each item becomes a point in space (an embedding ), and search means "find the nearest points to where the query lands."

An embedding is a list of numbers (a vector ) that captures the meaning of something — a word, a sentence, an image. An embedding model is trained so that similar inputs get similar vectors, meaning they point in nearly the same direction in space.

Real embeddings are big. OpenAI's text-embedding-3-small has 1536 numbers per item; all-MiniLM-L6-v2 has 384. You never read these numbers by hand — you let math compare them. In the worked example below, each item is just 3 numbers so you can see the whole idea at once: the first number is "how fruity," the third is "how techy."

The key intuition: direction encodes meaning . Two fruity items point the same way; a techy item points elsewhere. Search is "whose vector points most like my query's vector?"

To find "nearest," you need a way to score how close two vectors are. There are three you'll meet:

The cosine of the angle between vectors. Ignores length, cares only about direction. Ranges 0–1 for typical embeddings. The safe default for text.

Multiply matching components and sum. Factors in magnitude. Once vectors are L2-normalised (length 1), the dot product equals cosine — which is why databases normalise then use it (it's fast).

Straight-line distance between the points. Smaller is closer. Common for image features. Sensitive to magnitude, so normalise first.

The formula for cosine is dot(a, b) / (norm(a) * norm(b)) , where norm is the vector's length. That's exactly what the worked example codes up next.

Here is the heart of a vector database in pure Python — no libraries. You score the query against every item with cosine similarity, sort, and return the top-k. This is called brute force because it checks everything: simple and exact, but it does n comparisons. Read each comment, then run it.

Brute force is exact, but scoring every vector is O(n) . At 10 million vectors that's 10 million comparisons per query — too slow. The fix is approximate nearest neighbour (ANN) search: accept ~98% of the right answers in exchange for being 10–100× faster. You build an index once, then queries skip almost all the work.

HNSW (Hierarchical Navigable Small World) is the most popular ANN index. Picture a stack of maps:

Other index families: IVF (inverted file) splits vectors into clusters and searches only the nearest clusters; LSH hashes similar vectors into the same bucket; PQ (product quantization) compresses vectors to shrink memory. HNSW usually gives the best quality/speed tradeoff, which is why FAISS, Pinecone, Qdrant, and pgvector all offer it.

Pure similarity isn't always enough. You often need "find docs similar to this query but only from 2024 " or " only in English ." Vector databases let you attach metadata (a small dict of fields like topic , year , language ) to each vector and filter on it during search.

In the Chroma worked example below, the where= {' "topic": "fruit" '} clause keeps only fruit documents, then ranks those by similarity. Filtering shrinks the candidate set, which also speeds things up — a win on both relevance and latency.

The toy code above is what every production system does — just optimised and persisted. These two read-only examples show the real APIs. They won't run in the box (they need installs), so each ends with an # Expected output comment showing what you'd see.

❌ Wrong similarity metric → irrelevant results

Searching with Euclidean distance when your model was trained for cosine returns neighbours that look random.

✅ Fix: use the metric your embedding model expects (usually cosine / normalised inner product), and set the same metric on the index.

Feeding raw vectors into a dot-product (inner-product) index lets long vectors win regardless of direction.

✅ Fix: L2-normalise every embedding (and the query) to length 1, e.g. faiss.normalize_L2(x) , so dot product equals cosine.

❌ Expecting 100% accuracy from an approximate index

ANN indexes (HNSW, IVF, LSH) may miss a true neighbour now and then — that's the speed/accuracy tradeoff, not a bug.

✅ Fix: tune recall (e.g. HNSW efSearch , IVF nprobe ), or use a flat/brute-force index when exactness is mandatory and the collection is small.

Your query vector has a different length than the stored vectors — often from mixing two embedding models.

✅ Fix: embed queries and documents with the same model, and create the index with that exact dimension (e.g. 1536 for text-embedding-3-small ).

Support is faded now — only an outline is given. Put the whole loop together yourself: build a small collection, write cosine similarity, and return the top-k IDs for a query.

You can now explain what an embedding is, pick between cosine, dot product, and Euclidean, write a brute-force nearest-neighbour search by hand, reason about ANN and HNSW, filter by metadata, and choose between FAISS, Pinecone, Chroma, pgvector, and Weaviate.

🚀 Up next: Model Evaluation — measure how good your models and retrieval pipelines actually are with the right metrics.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary