RAG Systems

Learn how to ground a language model in your own documents — so it answers from real, retrievable facts instead of guessing or hallucinating.

Learn RAG Systems in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice exercise and a quick…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

A language model on its own is like a student taking a closed-book exam . It answers from memory — often brilliantly, but sometimes it confidently writes down something that simply isn't true. That confident-but-wrong answer is what we call a hallucination .

RAG turns it into an open-book exam. Before the student answers, you hand them the exact pages that contain the answer. They read those pages, then write a response grounded in real text. The same question, but now the answer is backed by a source you can point to.

Your job in RAG is to be the helpful librarian: given a question, find the right pages fast and slide them across the desk. The rest of this lesson is about how to do that finding well.

A language model only knows what it saw during training. That creates three real problems, and RAG fixes all three by feeding the model trusted text at question time.

Every RAG system is the same six steps. The first three happen once (when you load your documents); the last three happen on every question.

The clever part is step 4 — measuring which chunks are most similar in meaning to the question. The worked example below builds that whole loop by hand so you can see exactly what happens.

Run this. It builds a three-fact knowledge base, scores the question against each fact with cosine similarity, retrieves the best one, and stitches together the final prompt. Every line is commented and the expected output is at the bottom.

An embedding is a list of numbers — a vector — that captures the meaning of a piece of text. An embedding model is trained so that texts with similar meaning get vectors pointing in similar directions.

"Who created Python?" and "Python was made by Guido van Rossum" end up close together, even though they share almost no words. That's the magic: embeddings compare meaning , not spelling. This is also called semantic search — searching by sense rather than by keyword.

With cosine similarity . It measures the angle between two vectors and returns a score from -1 (opposite) to 1 (identical direction). A score near 1 means "very similar meaning". You already used it in the worked example — here's the formula it implements:

In real projects you never hand-write vectors. A model like all-MiniLM-L6-v2 produces a 384-number vector for any text in milliseconds. The example below shows the real call.

This is how embeddings look in production — read it to see the shape of the real API. (It needs the sentence-transformers package installed locally, so study it here rather than running it in the browser.)

Notice the pattern is identical to your hand-written loop: embed the chunks, embed the query, compare with cosine similarity, take the best. The only difference is a trained model makes the vectors instead of you.

One chunk is rarely enough. Most questions need facts spread across several chunks, so you keep the top-k — the k highest-scoring chunks, often the top 3 to 5. You score every chunk, sort highest-first, and slice off the best few.

Choosing k is a balance: too few and you miss relevant facts; too many and you bury the answer in noise and fill up the model's limited context window. Start around 3-5 and tune it. Run the worked example to see top-k selection in action.

Fill in the three blanks: complete the cosine-similarity formula, score every chunk, and find the index of the best one. The expected output is in the comments so you can check yourself.

Chunking is splitting documents into pieces small enough to embed and retrieve. It is the single biggest factor in RAG quality. Chunk badly and even a perfect model retrieves the wrong text.

Once you've retrieved the top-k chunks, you augment the prompt — paste the chunks in as context — and ask the model to generate an answer grounded in them. The instruction "answer using only the context" is what keeps it honest.

You don't build all six steps from scratch in real life. Libraries like LangChain and LlamaIndex handle chunking, embedding, and storage; a vector database like Chroma, FAISS, or Pinecone stores and searches the vectors fast. Read the pipeline below — it's the same six steps, just wired together with real tools and a Claude call for the final answer.

RAG can fail in two different places, so you measure two different things. Knowing which one is broken tells you what to fix.

Did the right chunks come back? Measure context recall (did we fetch the chunk that holds the answer?) and context precision (how much of what we fetched was actually relevant?).

Given the right chunks, was the answer good? Measure faithfulness (does the answer stick to the context, or did it drift?) and answer relevance (did it actually address the question?).

If retrieval is poor, improve chunking, the embedding model, or k . If retrieval is good but answers are bad, tighten the prompt and instruct the model to refuse when the context doesn't contain the answer. Tools like RAGAS automate these scores so you can track them as you change the pipeline.

Now put the back half of the pipeline together: sort the scored chunks, keep the top 2, and build a grounded prompt. Fill in each ___ and check against the expected output.

Support is fading now — you get the data and the brief, but the logic is yours to write. Build the whole retrieval step from scratch: cosine similarity, score every chunk, sort, and print the top 2. The expected output is in the comments.

These four mistakes account for most broken RAG systems. Learn to recognise them.

Chunks too small lose context; too large dilute the answer and overflow the context window.

✅ Fix: use a sensible size and a little overlap:

The model hallucinates because the wrong chunks came back — it never saw the answer.

✅ Fix: retrieve more candidates (and consider a reranker):

Stuffing too many chunks in blows past the model's context window — the request errors or silently drops text.

✅ Fix: keep k small and chunks reasonably sized; rerank to top 3-5:

Without telling the model to use only the context, it falls back on memory and you can't trace claims.

✅ Fix: instruct it to stay in the context and admit when it can't:

Lesson complete — you can build knowledge-grounded AI!

You now understand why RAG exists, the six-step pipeline, how embeddings and cosine similarity power retrieval, how chunking decides quality, and how to evaluate the whole thing. You even built the retrieval loop by hand.

🚀 Up next: Vector Databases — the engine that stores millions of embeddings and finds the top-k in milliseconds.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary