Neural Networks Introduction

By the end of this lesson you'll be able to compute a single neuron's output by hand, write ReLU and sigmoid in plain Python, and explain how layers learn by adjusting weights.

Learn Neural Networks Introduction in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Your brain has billions of neurons . Each one receives little electrical signals from its neighbours through connections called synapses . Some connections are strong and some are weak — they decide how much each incoming signal counts. When the combined signal crosses a threshold, the neuron fires and passes a signal on to the next neurons.

An artificial neuron copies this idea with arithmetic. The "synapse strengths" become numbers called weights , the firing threshold becomes a bias , and the "fire or not" decision becomes an activation function . Learning is just gradually turning the strength of each connection up or down until the whole network responds the way you want.

A neuron (also called a perceptron when it's on its own) is the smallest building block of a neural network. It takes some inputs and produces a single number. It does this in three tiny steps:

The weights and bias are the neuron's knowledge . Everything a network learns ends up stored as weights and biases. Here is a single neuron written in plain Python — read each comment, then run it.

The activation function is what makes a network powerful. It introduces non-linearity — a fancy way of saying "the output can bend and curve instead of being one straight line." Two activations cover almost everything a beginner needs:

Both are just a couple of lines of plain Python — the only thing you need from the standard library is math.exp for sigmoid.

One neuron can only draw a single straight boundary, which is too weak for most real problems. The fix is to use many neurons arranged in layers :

The forward pass is simply running data through the network from left to right: every neuron in a layer computes weighted sum + bias + activation, and its outputs become the inputs to the next layer. Stack enough layers and the network can approximate almost any function — that's the whole magic.

inputs → [hidden layer: many neurons] → [output layer] → prediction

A fresh network starts with random weights, so its first predictions are basically guesses. Training fixes that with a repeating loop:

That nudge size is controlled by the learning rate . Repeat this loop over thousands of examples and the weights slowly settle into values that make good predictions. You don't have to compute the gradients by hand — libraries like TensorFlow and PyTorch do it for you. Here's the same neuron idea written the professional way, plus a tiny Keras network, shown as a read-only reference.

These four mistakes trip up nearly everyone who builds their first network.

❌ No non-linearity = a glorified linear model

If every layer uses no activation (or only a linear one), stacking layers collapses into a single straight line. The network can't learn curves and will fail on problems like XOR.

✅ Fix: put a non-linear activation (ReLU) on every hidden layer:

Sigmoid and tanh flatten out for large inputs, so their slope (gradient) becomes almost 0. In deep networks the update signal shrinks to nothing and early layers stop learning.

✅ Fix: use ReLU in hidden layers; keep sigmoid for the final output only.

Feeding raw values on wildly different scales (e.g. age 0–100 next to salary 0–100000) makes training unstable — the big numbers dominate the weighted sum.

✅ Fix: scale features to a similar range before training.

If each weight update is too large, the network overshoots the good values and the loss bounces around or explodes to nan instead of going down.

✅ Fix: start small (e.g. 0.01) and only increase if learning is too slow.

Time to fly with less support. Build a 2-input neuron that ends with a sigmoid activation. Only a comment outline is given — fill in the logic yourself, then check against the expected output in the comments.

Lesson complete — you understand how neurons think!

You can now compute a neuron as weighted sum + bias + activation, write ReLU and sigmoid in plain Python, describe how layers stack into a forward pass, and explain how training nudges weights to shrink error. These are the exact foundations every deep learning model is built on.

🚀 Up next: Deep Learning Fundamentals — stack many layers, train with backpropagation, and tackle real datasets.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary