Classification Basics

By the end you'll predict yes/no categories with logistic regression — turning a score into a probability with the sigmoid, choosing a threshold, and judging the result with precision and recall instead of plain accuracy.

Learn Classification Basics in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice exercise and a…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

A thermometer gives you a continuous number — 72.3°F, 72.4°F, and every value in between. That's regression : the answer is a quantity on a sliding scale.

A traffic light gives you one of a few fixed states — red, amber, or green. That's classification : the answer is a label from a small set of choices.

Logistic regression is the bridge between them. Internally it computes a number (like the thermometer), but then it asks a yes/no question — "is this number high enough?" — to pick a label (like the traffic light). The sigmoid is how it converts the number into a confidence between 0 and 1.

Both are supervised learning — you train on examples that already have the right answer. The only difference is the kind of answer you want back.

House price, tomorrow's temperature, expected sales. The output is continuous: any value on a scale.

Spam / not spam, cat / dog, pass / fail, disease / healthy. The output is one category from a fixed set.

This lesson covers binary classification — exactly two classes, which you label 1 (the "positive" class you care about, e.g. spam) and 0 (the negative class). Everything you learn extends naturally to more classes later.

Plain linear regression can output any number — minus a million, plus a million. But a probability has to live between 0 and 1. The sigmoid function fixes that: it takes any number z and squashes it onto the range 0 to 1 with a smooth S-shaped curve.

In logistic regression, z is the same weighted sum as linear regression — w1*x1 + w2*x2 + ... + bias — and the sigmoid turns that score into a probability. Run the worked example and watch the bar grow from empty to full as z climbs.

A probability like 0.73 isn't a label yet. To get a label you pick a threshold — a cut-off. The default is 0.5 : if the probability is at or above it, predict class 1; otherwise class 0.

The set of feature values where the probability is exactly the threshold is the decision boundary . For logistic regression with a 0.5 threshold, that's wherever the score z = 0 — because sigmoid(0) = 0.5 . On one side of the line the model says 1, on the other it says 0.

The threshold is a dial you control. Lower it (say to 0.3) and the model flags more positives — it catches more real ones but also raises more false alarms. Raise it (say to 0.7) and it flags fewer — fewer false alarms but more misses. You'll use that dial deliberately in the mini-challenge.

The worked example below does the full pipeline by hand: weighted sum → sigmoid → threshold. Read each row and confirm the prediction matches the probability.

Finish two blanks: complete the sigmoid formula, then compare each probability against the threshold to pick a class. The expected output is in the comments so you can self-check.

To judge a classifier you compare its predictions to the truth and sort every result into four buckets — the confusion matrix :

From those four counts come the four metrics you'll use constantly:

The confusion counts are already worked out for you in the loop. Fill in the two metric denominators using the formulas above, then run it and match the expected output.

You've now built every piece by hand, so the library version will make complete sense. In real projects you use scikit-learn ( sklearn ): LogisticRegression learns the weights and bias for you with .fit() , and ready-made functions compute the metrics — so you don't re-implement and re-test the maths yourself.

The example below needs scikit-learn installed, so it's shown as a read-along with the expected output in the comments. Run it locally after pip install scikit-learn to see it live.

Notice the shape: model.fit(X, y) learns, model.predict(X) gives labels, model.predict_proba(X) gives probabilities, and the metric functions take (truth, predictions) . Same four metrics you just coded by hand.

These four mistakes trip up almost everyone learning classification. Spot them early.

When one class is rare, accuracy looks great while the model is useless:

✅ Fix: report precision and recall (or F1) on the positive class, not accuracy alone.

Leaving the threshold at 0.5 when your problem isn't balanced. A cancer screen at 0.5 may miss real cases:

✅ Fix: lower the threshold (e.g. 0.3) to raise recall when misses are costly; raise it to favour precision.

Measuring performance on the same rows you trained on (or letting test data peek into training) gives a fake-perfect score:

✅ Fix: split your data — train on one part, evaluate on a held-out test set the model never saw ( train_test_split ).

Swapping the denominators. The fix is to anchor on what each one asks:

✅ Fix: precision is about your predictions (add FP); recall is about the actual positives (add FN).

Now write it from a blank slate — only a comment outline is given. Build a tiny screener that deliberately uses a low threshold of 0.3 to favour recall, because in screening a miss is worse than a false alarm. The expected output is in the comments so you can check yourself.

Lesson 5 complete — you can classify!

You can tell classification from regression, turn a score into a probability with the sigmoid, draw a decision boundary at a threshold you choose, and judge a model with accuracy, precision, recall and F1 — by hand and with scikit-learn's LogisticRegression .

🚀 Up next: Decision Trees — the most interpretable classifier, splitting data into simple yes/no questions.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary