Data Preprocessing
Turn messy, raw data into clean, model-ready numbers — handle missing values, scale features, encode categories, and split safely without leaking the answers.
Learn Data Preprocessing in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a practice exercise and a…
Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.
Imagine cooking with unwashed vegetables, ingredients still in their packaging, and spices measured in totally different units — pinches, kilograms, cups. The meal would be a disaster, no matter how good the recipe is.
Data preprocessing is the prep work before cooking. Filling missing values is like replacing the one carrot that went bad. Scaling is putting every ingredient on the same measuring scale so none overwhelms the dish. Encoding is translating "a handful of basil" into an exact number the recipe can use. And splitting train/test is tasting with a clean spoon you never dipped back in the pot — so your judgement stays honest.
⚠️ Golden Rule: "Garbage in, garbage out." Data scientists spend 60–80% of their time here because the best algorithm on earth cannot rescue badly prepared data.
A missing value is a hole in your data — a survey question left blank, a sensor that dropped out, a field that was never filled in. In Python you'll see these as None (or NaN in pandas). Most ML algorithms crash or misbehave if you feed them holes, so you must deal with every one.
Read the worked example below line by line, then run it. Every line states the result in a comment.
Scaling means rewriting every feature so they all cover a similar range. Why bother? Imagine one column is age (20–60) and another is salary (35,000–140,000). Many algorithms measure distance between numbers — and salary's huge values would completely dominate, drowning out age entirely. Scaling levels the playing field.
The smallest value becomes 0, the largest becomes 1. Great for neural networks and any bounded input.
Re-centres data around 0. The "std" (standard deviation) measures spread. Great for linear models, SVMs, and k-NN.
A categorical feature is text that names a group — a city, a plan, a colour. ML maths needs numbers, so you must translate categories into numbers. How you translate depends on whether the categories have an order.
To know if a model actually learned (rather than just memorised), you hold back some data. You train on most of it (commonly 80%) and test on the rest (20%) — data the model has never seen. Always shuffle first so the split isn't accidentally ordered by date or class.
Data leakage is the silent killer of ML projects. It happens whenever information from the test set sneaks into training. The most common cause is scaling before splitting: if you compute the min and max over the whole dataset, those numbers already "know" about your test rows.
You just implemented every step by hand so you understand exactly what happens. In real projects you'd reach for pandas and scikit-learn , which do the same work in a handful of lines — and crucially, enforce the "fit on train, transform on test" rule for you.
Notice the pattern repeated four times: fit_transform on train, then transform on test. That single discipline is what keeps your evaluation honest.
Time to fly with less support. Fill in the missing temperatures using the median of the values that are present. The starter below is just a comment outline — write the logic yourself.
These four mistakes quietly wreck more ML projects than any algorithm bug. Learn to spot them.
Fitting the scaler on the whole dataset lets the test set influence training. Scores look great, then collapse in production.
✅ Fix: split first, fit on train, apply to test:
Calling fit (or recomputing min/max) on test data is leakage in disguise — the test set is supposed to be unseen.
If 30% of rows have one hole each, dropping them throws away a third of your data — and may bias what's left.
✅ Fix: impute (fill) instead, drop only when <5% affected:
Mapping unordered categories to 0,1,2 invents a fake ranking the model will believe.
✅ Fix: one-hot encode anything without a natural order:
Reserve label encoding for genuinely ordinal data (low/medium/high).
You can now spot and fill missing values, scale features with min-max and z-score, encode categories the right way, and split data without leaking it into your evaluation. You implemented every step in plain Python, so the pandas and scikit-learn versions will feel like shortcuts rather than magic.
🚀 Up next: Linear Regression — feed your freshly cleaned data into your first real machine-learning model and watch it make predictions.
Practice quiz
When should you fit a scaler relative to the train/test split?
- Fit on all data before splitting
- Fit on the test set only
- Fit on the training set only, after splitting
- Fit separately on train and test
Answer: Fit on the training set only, after splitting. Split first, fit the scaler on training data only, then apply it to the test set — fitting on all data leaks test info.
What is the min-max scaling formula?
- (x - min) / (max - min)
- (x - mean) / std
- x / 255
- (x - max) / (max - min)
Answer: (x - min) / (max - min). Min-max scaling maps values into 0..1 using (x − min) / (max − min): the smallest becomes 0, the largest becomes 1.
After standardisation (z-score), a feature has:
- Min 0 and max 1
- All values between -1 and 1
- Sum equal to 1
- Mean 0 and standard deviation 1
Answer: Mean 0 and standard deviation 1. Standardisation uses (x − mean) / std, re-centring the data to mean 0 with a standard deviation of 1.
Which encoding suits ORDINAL categories like low < medium < high?
- One-hot encoding
- Label encoding
- Min-max scaling
- Median imputation
Answer: Label encoding. Label encoding assigns one number per category and is appropriate when the categories have a real order.
Why is label-encoding nominal categories (London, Paris, Berlin) a mistake?
- It invents a fake ranking the model will believe
- It uses too much memory
- It crashes scikit-learn
- It deletes rows
Answer: It invents a fake ranking the model will believe. If London=0 and Paris=1, the model wrongly infers Paris > London. Use one-hot encoding for unordered categories.
A one-hot encoded row for one of three categories looks like:
- A single integer 0, 1, or 2
- A probability between 0 and 1
- A list with a single 1 and the rest 0
- The category's text unchanged
Answer: A list with a single 1 and the rest 0. One-hot encoding makes one column per category, with a single 1 in the matching slot and 0 elsewhere.
Which fill strategy is more robust for skewed numeric data?
- Mean
- Median
- Mode of the text
- Always drop the row
Answer: Median. The median (middle value) resists outliers, whereas a few huge values pull the mean upward in skewed data.
When is it reasonably safe to simply drop rows with missing values?
- When most rows are affected
- Whenever the column is numeric
- Never, under any circumstances
- When only a very small fraction (roughly under 5%) are affected
Answer: When only a very small fraction (roughly under 5%) are affected. Dropping is acceptable only when very few rows are affected; otherwise you throw away signal and may bias the data.
What is data leakage in preprocessing?
- When the model runs out of memory
- When information from the test set influences training
- When features have different units
- When categories are unordered
Answer: When information from the test set influences training. Leakage is any case where test-set information sneaks into training — e.g. computing scaling stats over all data first.
What is the correct order to avoid leakage?
- fit → split → transform
- transform → split → fit
- split → fit (on train) → transform
- fit → transform → split
Answer: split → fit (on train) → transform. Split first, fit the scaler/encoder on the training set only, then transform both train and test with it.