Generative Models: Autoencoders, VAEs & GANs

By the end of this lesson you'll understand how machines create brand-new data — and you'll have written a working text generator that samples a distribution it learned itself.

Learn Generative Models: Autoencoders, VAEs & GANs in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Picture an artist who studies hundreds of Van Gogh paintings. They never copy a single one — instead they absorb the style : the swirling skies, the thick brushstrokes, the colour choices. Afterwards they can paint a brand-new scene that has never existed, yet unmistakably looks like a Van Gogh.

That is exactly what a generative model does. During training it watches a mountain of real examples and learns the underlying data distribution — the statistical "style" of the data. During generation it samples from that learned distribution to produce something new that fits the style but isn't a copy.

Keep this picture in mind: learn a style → create new work . Everything below is just different machinery for doing those two steps.

Most models you've met so far are discriminative : you hand them data and they predict a label. A spam filter learns the line between "spam" and "not spam". In maths, it estimates P(label | data) — the probability of a label given the input.

A generative model is more ambitious. Instead of drawing a boundary, it learns what the data itself looks like — P(data) . Because it understands the whole shape of the data, it can sample brand-new examples: a new sentence, a new face, a new song.

"Is this email spam?" — it sorts existing things into bins.

"What does a typical email look like?" — it can write a new one.

At its heart, generation = sampling . A generative model is really just a probability distribution you can draw from. Run the worked example below: it samples colours whose probabilities were "learned", then checks that the samples echo those probabilities.

Read every comment, then hit run. Notice the output counts roughly match the learned weights — that's the model reproducing the style of its data.

An autoregressive model generates a sequence by predicting the next item from everything before it, then feeding that prediction back in. GPT writes a sentence one token at a time this way. The simplest version is a Markov chain : it only looks at the current word to guess the next.

The worked example below learns which words follow which from a four-line corpus, then generates new sentences by repeatedly sampling. It is a complete generative model in under 30 lines — no maths library required.

Almost every modern generator belongs to one of four families. They differ in how they learn the distribution and sample from it.

📝 Autoregressive (GPT, PixelCNN, Markov chains)

Generate one element at a time, conditioning on what came before. Great for text; slower for long sequences because each step depends on the last.

Squeeze data into a smooth latent space and decode it back. Stable to train and easy to interpolate, but outputs tend to be blurry.

A generator forges data while a discriminator tries to catch fakes; they improve by competing. Produces sharp, realistic images but is notoriously tricky to train.

Start from pure noise and denoise step by step until an image appears. Highest quality today; generation is slower because it takes many steps.

Latent space is a compressed map of everything the model can create. Each point on the map is a recipe for one output; nearby points produce similar outputs. A face generator might use the same direction in latent space to mean "add glasses" or "make older".

Because the space is continuous, you can interpolate — walk in a straight line from the point for a cat to the point for a dog and decode every step along the way. The outputs morph smoothly from cat to dog. That smoothness is exactly why VAEs are loved for exploration, even when their images are a little soft.

Evaluation is genuinely hard: there is no single "correct" output to compare against. You instead balance two things — quality (do samples look real?) and diversity (do they cover the whole range of real data, or only a few favourites?).

The most common image metric is FID (Frechet Inception Distance). It compares the statistics of generated images to real images in a learned feature space — lower is better . FID rewards models that are both realistic and varied, which is why a model that produces one perfect image over and over still scores badly.

For text, you'll see perplexity and human preference scores; for everything, a human eye remains the final judge.

Fill in the blanks so the sampler draws from a weather distribution. The skeleton is done — you only supply the weights and the comparison.

The model has already learned its transition table. Finish the generation loop so it samples one new word at a time, like an autoregressive model.

Support is faded now — you get an outline only. Build a Markov generator from scratch over your own corpus.

The generator finds one output that always fools the discriminator and produces only that — endless near-identical faces. Diversity drops to almost nothing.

✅ Fix: add minibatch discrimination, use a Wasserstein loss (WGAN-GP), or unroll the discriminator so the generator can't exploit a single weakness.

VAEs minimise average reconstruction error, so when several outputs are plausible they hedge by averaging them — and the average of sharp images is blurry.

✅ Fix: use a perceptual loss, a VAE-GAN hybrid, or a discrete latent (VQ-VAE) to commit to specific outputs instead of averaging.

Adversarial losses oscillate or diverge — the generator and discriminator chase each other instead of converging, and the loss curve never settles.

✅ Fix: balance the learning rates (two-timescale rule), add spectral normalisation, and keep the discriminator from becoming too strong too fast.

❌ "It looks fine but the number says it's bad" — evaluation difficulty

There's no single correct output, so loss going down doesn't guarantee good samples. A model can score a low loss while still producing repetitive or unrealistic results.

✅ Fix: report FID (lower is better) and a diversity check, then back it up with human evaluation — never trust one metric alone.

You can now tell discriminative from generative models, name the four families, explain latent space and interpolation, sample from a learned distribution, and read evaluation metrics like FID. Best of all, you built a working autoregressive text generator in plain Python.

🚀 Up next: Diffusion Models — the noise-to-image technique behind DALL·E and Stable Diffusion, and today's quality leader.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary