AutoML & Neural Architecture Search

What does AutoML automate?

• Writing the dataset by hand
• Drawing charts
• Searching across algorithms and hyperparameters to find a good model
• Labelling the data

Answer: Searching across algorithms and hyperparameters to find a good model. AutoML systematically searches the space of algorithms and their hyperparameters instead of you trying each by hand.

What is hyperparameter optimisation (HPO)?

• Tuning settings like learning rate or tree depth that are not learned from data
• Learning the model weights
• Collecting more data
• Deploying the model

Answer: Tuning settings like learning rate or tree depth that are not learned from data. HPO tunes configuration values (e.g. n_estimators, max_depth, C) that control training but aren't learned by the model.

Why is Bayesian optimisation smarter than pure random search?

• It tries every combination exhaustively
• It ignores past results
• It only works on images
• It builds a model of the objective and focuses on promising regions

Answer: It builds a model of the objective and focuses on promising regions. Bayesian optimisation uses prior trial results to model the objective and concentrate search where improvement is likely.

What does Neural Architecture Search (NAS) automate?

• Cleaning the data
• The design of the neural network architecture itself
• Writing the loss function only
• Choosing the cloud provider

Answer: The design of the neural network architecture itself. NAS discovers the network's layers and connections automatically instead of a human hand-designing the architecture.

Which NAS approach is much faster than RL-based search by being differentiable?

• DARTS
• NASNet
• Grid search
• Random forest

Answer: DARTS. DARTS (Differentiable Architecture Search) relaxes the search to be differentiable, making it far faster than RL-based NAS.

Why can too many search iterations be harmful?

• They make the data larger
• They delete the model
• They can overfit to the validation/CV folds
• They always reduce accuracy to zero

Answer: They can overfit to the validation/CV folds. Running an enormous search can tune the model to quirks of the validation folds, overfitting the selection itself.

Why does 'garbage in, garbage out' still apply to AutoML?

• AutoML cleans data automatically
• AutoML can't fix poor features or dirty data — you must prepare them first
• AutoML only works on text
• It does not apply to AutoML

Answer: AutoML can't fix poor features or dirty data — you must prepare them first. AutoML searches models, but it cannot rescue badly prepared data or missing feature engineering.

A practical way to use AutoML well is to:

• Ship its first result without review
• Avoid validation entirely
• Only use it for deep learning
• Use it for strong baselines, then manually iterate on the top candidates

Answer: Use it for strong baselines, then manually iterate on the top candidates. AutoML is great for baselines; experts then hand-tune the top 2-3 candidates it surfaces.

Full RL-based NAS (like NASNet) is mainly limited by:

• Lack of any datasets
• Its very large compute (GPU-hours) budget
• Being unable to use convolutions
• Requiring no training at all

Answer: Its very large compute (GPU-hours) budget. RL-based NAS can need tens of thousands of GPU-hours, which is why faster methods like DARTS/ENAS are preferred in practice.

Which tool is a popular general-purpose hyperparameter optimisation library?

• Pillow
• Matplotlib
• Optuna
• Requests

Answer: Optuna. Optuna is a flexible HPO framework usable with any model; Auto-sklearn and AutoGluon are AutoML systems, DARTS is for NAS.