Reinforcement Learning

Teach an agent to make good decisions through trial and error. By the end you'll understand the RL loop, write an epsilon-greedy agent in plain Python, and know where RL powers real systems — from game AI to robots.

Learn Reinforcement Learning in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice exercise and…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

You can't hand a puppy a textbook. Instead you say "sit", wait, and the moment it sits you give a treat. The treat is a reward . The puppy doesn't know why the treat came at first — but after many tries it links the situation ("I heard 'sit'") and the action ("I sat") to the reward.

That is reinforcement learning exactly. The puppy is the agent . Your living room is the environment . "I just heard the word sit" is the state . Sitting, lying down, or running off are the actions . The treat (or no treat) is the reward . Over time the puppy forms a policy : "when I hear 'sit', sitting pays off."

Notice there were no labelled examples — nobody showed the puppy ten thousand photos of correct sitting. It learned purely from interaction and feedback . That's what separates RL from supervised learning.

Every RL problem is built from the same five pieces. Learn these names once and the rest of the field reads easily.

Jargon check: a policy is the agent's strategy — a rule mapping each state to an action. The agent's job is to improve its policy until it reliably collects high reward.

RL runs in a loop: the agent observes a state , picks an action , the environment returns a new state and a reward , and round it goes. One full run from start to finish is called an episode .

Read this fully-commented example. It builds a tiny 5-cell corridor by hand and walks an agent to the goal so you can see every part of the loop with your own eyes. Run it and match the output.

The -1 per step plus +10 at the goal is called reward shaping — designing the numbers so the agent prefers short paths. Get the shaping wrong and the agent learns the wrong thing.

There are two quantities an RL agent can learn, and beginners mix them up constantly:

A rule that maps a state to an action. Ask it "I'm in state X, what now?" and it answers with an action. The puppy's "hear sit → sit" is a policy.

A number estimating the total future reward you expect from a state (or a state-action pair). It doesn't tell you what to do directly — it scores options.

Many algorithms learn values first, then act greedily on them: "estimate how good each action is, then pick the highest." That is exactly what the bandit below does — it keeps a value estimate per arm and exploits the best one.

Here is the central dilemma of RL. Exploitation means choosing the action you currently think is best. Exploration means trying a different action to find out if something better exists. Lean too far either way and you lose.

The classic toy problem is the multi-armed bandit : several slot machine arms, each paying out with a hidden probability. The epsilon-greedy rule solves it simply — with probability epsilon (say 0.1) explore a random arm, otherwise exploit the arm with the highest current estimate.

Because most pulls exploit the best-known arm, arm 2 (the true best) gets pulled thousands of times — but the occasional 10% exploration is what let the agent find arm 2 in the first place. That balance is the whole game.

The Markov Decision Process is the formal frame underneath every RL problem. Don't let the name scare you — it's just the five pieces written precisely:

The "Markov" property is the key simplifying assumption: the next state depends only on the current state and action — not on the entire history of how you got there . That makes the maths tractable. Your corridor example was a small MDP, and so was the bandit (a one-state MDP).

You built tiny environments by hand to learn the mechanics. In real projects you reach for Gymnasium (the maintained successor to OpenAI Gym), which provides ready-made environments behind a standard reset() / step() interface — the very same loop you wrote.

This block needs pip install gymnasium , so run it on your own machine rather than in the sandbox above. Read it to see how the hand-built loop maps onto a real library.

The explore branch is written for you. Fill in the exploit line so the agent picks the arm with the highest estimate. Match the expected output in the comment.

Now write both halves of the decision: the explore return (a random arm) and the exploit return (the best arm). The hints are right there in the comments.

Support is fading now. You get only a comment outline — write the epsilon-greedy agent yourself. Use the two worked examples above as your reference.

A pure-greedy agent locks onto whichever arm happened to win first and never discovers the truly best one.

✅ Fix: keep a small exploration rate (e.g. epsilon = 0.1 ), or decay it from high to low over time so you explore early and exploit later.

If you reward the agent for the wrong thing, it optimises the wrong thing. A vacuum bot rewarded for "dust collected" learns to dump dust and re-vacuum it.

✅ Fix: reward the outcome you actually want , not a proxy. Add small penalties (like -1 per step) to discourage stalling.

If reward only arrives at a distant goal and is zero everywhere else, the agent wanders randomly and almost never stumbles onto it, so it never learns.

✅ Fix: add intermediate rewards (shaping), shrink the environment while learning, or use exploration bonuses to encourage reaching new states.

If the environment changes over time (payouts drift, an opponent adapts), an estimate built as a plain average of all past rewards reacts too slowly.

✅ Fix: use a fixed step size — estimate += alpha * (reward - estimate) — so recent experience counts more than ancient experience.

Pro tip: ChatGPT and other chat models are fine-tuned with Reinforcement Learning from Human Feedback (RLHF). Humans rank responses, and RL nudges the model toward the answers people prefer — the exact same reward-maximising loop you learned here, just with a language model as the agent.

You now know the language of reinforcement learning: an agent acts in an environment , observing states , choosing actions , and collecting rewards in a loop. You can explain policy vs value , balance exploration vs exploitation with epsilon-greedy, frame a problem as a Markov Decision Process , and name where RL is used.

🚀 Up next: Model Deployment — take a trained model out of your notebook and put it into production.

Practice quiz

What is reinforcement learning in simple terms?

  • Learning from labelled examples
  • Memorising a fixed dataset
  • Learning by trial and error: an agent acts, receives rewards, and learns a policy that maximises total reward
  • Compressing data

Answer: Learning by trial and error: an agent acts, receives rewards, and learns a policy that maximises total reward. RL learns from interaction and a reward signal, not from labelled correct answers.

How does RL differ from supervised learning?

  • RL has no labels — it only gets a reward signal after acting and must discover good actions itself
  • RL uses labelled examples; supervised does not
  • They are the same
  • RL never uses feedback

Answer: RL has no labels — it only gets a reward signal after acting and must discover good actions itself. Supervised learning is told the right answer; RL only sees rewards and figures out which actions are good.

Which are the five core pieces of an RL problem?

  • Input, hidden, output, loss, optimizer
  • Train, test, validate, score, deploy
  • Chunk, embed, store, retrieve, generate
  • Agent, environment, state, action, reward

Answer: Agent, environment, state, action, reward. Every RL problem is built from agent, environment, state, action, and reward.

What is a policy?

  • A number scoring a state
  • The agent's strategy — a rule mapping each state to an action
  • The reward signal
  • The environment's transition table

Answer: The agent's strategy — a rule mapping each state to an action. A policy tells the agent what action to take in each state.

What is the difference between a policy and a value?

  • A policy says what to do; a value estimates the expected future reward of a state or action
  • A value says what to do; a policy is a reward
  • They are identical
  • A policy is always a single number

Answer: A policy says what to do; a value estimates the expected future reward of a state or action. Policy answers 'what should I do?'; value answers 'how good is this?' (expected future reward).

What is the exploration vs exploitation trade-off?

  • Choosing between two models
  • Splitting data into train and test
  • Exploitation picks the action you think is best; exploration tries others to find something better
  • Tuning the learning rate

Answer: Exploitation picks the action you think is best; exploration tries others to find something better. Too little exploration gets stuck on a mediocre choice; too much wastes reward. Epsilon-greedy balances them.

How does the epsilon-greedy rule choose actions?

  • Always pick the best-estimated action
  • With probability epsilon pick a random action, otherwise pick the highest-estimated action
  • Always pick randomly
  • Pick the lowest-estimated action

Answer: With probability epsilon pick a random action, otherwise pick the highest-estimated action. Epsilon controls how often the agent explores instead of exploiting the current best estimate.

In a multi-armed bandit, what is the agent trying to discover?

  • The number of arms
  • The colour of each arm
  • The exact payout formula given to it
  • Which arm has the highest hidden payout probability, using only the rewards it observes

Answer: Which arm has the highest hidden payout probability, using only the rewards it observes. The agent never reads the true probabilities; it estimates each arm's value from observed rewards.

What is a Markov Decision Process (MDP)?

  • A neural network architecture
  • The formal frame for RL: states, actions, transition probabilities, rewards, and a discount
  • A type of reward only
  • A clustering algorithm

Answer: The formal frame for RL: states, actions, transition probabilities, rewards, and a discount. An MDP defines states, actions, transitions, rewards, and gamma — the formal structure under every RL problem.

What does the 'Markov' property mean?

  • The agent remembers every past step
  • Rewards are always positive
  • The next state depends only on the current state and action, not the full history
  • The environment never changes

Answer: The next state depends only on the current state and action, not the full history. The Markov property says the future depends only on the present state and action, which makes the maths tractable.