Transformers & LLMs

By the end of this lesson you'll be able to explain — and hand-compute — the self-attention mechanism that powers ChatGPT, BERT, and every modern language model.

Learn Transformers & LLMs in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice exercise and a…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Picture yourself at a noisy dinner table. Someone says "can you pass it?" To understand "it" , you don't weigh every word equally — you instinctively focus on the salt someone mentioned a moment ago and mostly ignore the unrelated chatter.

Self-attention does exactly this. For each word, the model asks "which other words should I focus on to understand this one?" It hands out a budget of attention — a lot to the few words that matter, a little to the rest — and the budget always adds up to 100%. That is the whole idea you'll build piece by piece below.

Every token (word piece) is turned into three vectors. A Query is what the token is looking for. A Key is what each token advertises about itself. A Value is the actual information a token carries.

To work out how much word A should attend to word B, you compare A's Query with B's Key. The comparison is a dot product — multiply the matching numbers and add them up. A bigger dot product means a better match, so more attention.

The raw dot product can grow large when vectors are long, which makes the next step (softmax) unstable. The fix is to divide by the square root of the vector length, written d_k . That single division is why it's called scaled dot-product attention.

Here it is with no libraries — just two small vectors and one score:

A real token produces one score for every other token. Softmax converts that list of scores into weights between 0 and 1 that add up to exactly 1.0 — your 100% attention budget. The highest score gets the biggest slice.

This worked example softmaxes three scores by hand:

Fill in the blank so the program prints the dot-product score between the query and key.

In real models you don't loop in Python — you use matrix operations so the whole sequence is processed at once. The maths is identical: scores, softmax, weighted sum of Values. The @ symbol is matrix multiplication.

Note the shapes: a 3-token input produces a 3×3 score matrix (every token scored against every token), and the output keeps the original shape so blocks can stack.

One attention calculation can only learn one kind of relationship. Multi-head attention runs several attention computations in parallel — each "head" gets its own learned projection of the data, so one head might track grammar while another tracks meaning. Their outputs are concatenated and mixed back together.

It's like proofreading a sentence three times — once for spelling, once for grammar, once for tone. Same sentence, different lenses, richer understanding.

Attention treats the input as an unordered set . On its own it can't tell "dog bites man" from "man bites dog" . Positional encoding fixes this by adding a unique pattern to each position before attention runs.

The original paper used sine and cosine waves of different frequencies, so each position gets a distinct fingerprint and nearby positions stay similar. Newer models often use learned or rotary positions, but the goal is the same: tell the model where each token sits .

A transformer block bolts four pieces together, and real models stack dozens of these blocks:

Here's a complete block in PyTorch. Notice the input and output shapes are identical — that's what lets you stack blocks:

Fill in the two blanks so the weights sum to exactly 1.0 when rounded to one decimal place.

GPT and BERT use the same building blocks but wire attention differently:

Reads the whole sequence at once and looks both left and right (bidirectional). Great for understanding : classification, search, filling blanks.

Generates one token at a time and may only look at earlier tokens (masked/causal attention). Great for generating text.

Translation models often use both: an encoder to read the source and a decoder to write the translation.

You almost never train a transformer from scratch. Pre-trained models from Hugging Face give you a working LLM instantly — a decoder generates, an encoder understands:

Fine-tuning a pre-trained model beats training from scratch in almost every real project.

❌ RuntimeError: mat1 and mat2 shapes cannot be multiplied

Your Query and Key have mismatched dimensions, so the score matrix can't be formed.

✅ Fix: make sure Q and K share the last dimension, and transpose K:

You applied softmax along the wrong axis, so it normalised across the wrong tokens.

✅ Fix: softmax over the last dimension (each row of scores):

✅ Fix: subtract the max before exp, and scale by √d_k:

❌ AssertionError: embed_dim must be divisible by num_heads

Multi-head attention splits d_model evenly across heads.

✅ Fix: pick heads that divide the model size (e.g. d_model=8, n_heads=2 or 4).

Put the whole pipeline together: softmax three scores into weights, then take the weighted sum of three Value vectors to produce one context vector. The starter has only a comment outline — you write the logic.

You can now explain self-attention with the Query/Key/Value intuition, hand-compute a scaled dot-product score, softmax scores into attention weights, and describe multi-head attention, positional encoding, the transformer block, and how encoders (BERT) differ from decoders (GPT). That's the core of every modern language model.

🚀 Up next: Reinforcement Learning — how AI agents learn from trial and error.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary