Your one-stop platform to prepare, practice and ace your interviews.
Loading your questions…
41 hand-picked ML Fundamentals interview questions with detailed answers. Open the interactive version above to search, filter by difficulty, run code, bookmark questions and track your progress.
Training — the model learns patterns from data by adjusting weights to minimise a loss function. Computationally expensive; done offline or periodically.
Inference — using the trained model to make predictions on new, unseen data. Must be fast and reliable for production serving.
Training uses backward propagation and optimisers (Adam, SGD). Inference is forward-pass only.
Classification — predict a category (discrete label). Binary (spam/not spam) or multi-class (cat/dog/bird). Output: class probabilities.
Regression — predict a continuous number. Example: house price, temperature, revenue forecast. Output: a real-valued number.
Same algorithms can do both with different loss functions and output layers.
Overfitting — model memorises training data (including noise) and fails on new data. Signs: high training accuracy, low validation accuracy. Fixes: more data, regularisation, simpler model, dropout, early stopping.
Underfitting — model is too simple to capture patterns. Signs: poor performance on both training and validation. Fixes: more features, complex model, train longer.
Goal: generalisation — good performance on unseen data.
Total prediction error = bias² + variance + irreducible noise.
Increasing model complexity reduces bias but increases variance — find the sweet spot.
Using test data during development causes data leakage — you overfit to the test set indirectly.
from sklearn.model_selection import train_test_split
X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.3)
X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5)Split data into k folds. Train on k-1 folds, validate on the remaining fold. Repeat k times, each fold serving as validation once. Average the k scores for a robust performance estimate.
Useful when data is limited — every sample gets to be in both training and validation. Common: k=5 or k=10.
Stratified k-fold preserves class distribution in each fold (important for imbalanced data).
from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5, scoring='f1')
print(scores.mean(), scores.std())The loss function defines what 'wrong' means — choose one aligned with your business metric.
An optimisation algorithm that iteratively adjusts model weights in the direction that reduces the loss (opposite to the gradient).
A network of layers of neurons connected by weighted edges:
Training: forward pass (compute output) → compute loss → backward pass (backpropagation computes gradients) → update weights.
max(0, x). Default for hidden layers. Fast, avoids vanishing gradient. Dead ReLU problem (neurons output 0 forever).| Architecture | Strength | Typical use |
|---|---|---|
| CNN | Spatial/local patterns via convolutions | Image classification, object detection, medical imaging |
| RNN/LSTM | Sequential data, temporal dependencies | Time series, early NLP (largely replaced) |
| Transformer | Long-range dependencies, parallel training | NLP, vision (ViT), multimodal, LLMs |
Transformers dominate NLP and are increasingly used for vision, audio, and tabular data too.
Transforming raw data into informative inputs the model can learn from. Examples:
Deep learning reduces manual feature engineering but preprocessing still matters.
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test) # use train stats!TP / (TP + FP). "When I say yes, how often am I right?"TP / (TP + FN). "Of all real positives, how many did I find?"Choose based on cost: spam filter needs high precision (don't block real emails); cancer screening needs high recall (don't miss cases).
from sklearn.metrics import precision_score, recall_score, f1_score
print(f1_score(y_true, y_pred))A table comparing predicted vs actual labels:
Predicted
Pos Neg
Actual Pos TP FN
Neg FP TNAll classification metrics derive from these four counts.
from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_true, y_pred))ROC curve — plots True Positive Rate (recall) vs False Positive Rate at various classification thresholds.
AUC (Area Under Curve) — single number summarising ROC. 1.0 = perfect; 0.5 = random guessing.
Threshold-independent metric — useful for comparing models regardless of the chosen decision threshold. Good for imbalanced datasets when you haven't decided on a threshold yet.
from sklearn.metrics import roc_auc_score
auc = roc_auc_score(y_true, y_prob) # needs probability scores, not class labelsHyperparameters are set before training (learning rate, tree depth, number of layers) — not learned from data.
Methods:
Always tune on validation set, evaluate final config on test set.
from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(model, param_grid, cv=5, scoring='f1')
grid.fit(X_train, y_train)
print(grid.best_params_)Using a model pre-trained on a large dataset as a starting point for your specific task. Instead of training from scratch, you fine-tune the last layers (or the whole model) on your smaller dataset.
Examples: ImageNet-pretrained ResNet for medical imaging; BERT/GPT for NLP tasks; Whisper for speech.
Saves data, compute, and time — almost always better than training from scratch on small datasets.
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained(
"bert-base-uncased", num_labels=3
)When information from outside the training set leaks into training, giving unrealistically good evaluation metrics that don't hold in production.
Common causes:
Fix: split first, then preprocess. Use pipelines. Audit features for leakage.
from sklearn.pipeline import Pipeline
pipe = Pipeline([
('scaler', StandardScaler()),
('model', LogisticRegression())
])
pipe.fit(X_train, y_train) # scaler fits only on trainclass_weight='balanced').from sklearn.utils.class_weight import compute_class_weight
weights = compute_class_weight('balanced', classes=np.unique(y), y=y)Combine multiple models for better performance than any single model:
Gradient boosted trees dominate tabular data competitions and production ML.
import xgboost as xgb
model = xgb.XGBClassifier(n_estimators=100, max_depth=6)
model.fit(X_train, y_train)Both add a penalty term to the loss function to prevent large weights:
In neural nets, L2 regularisation is called weight decay.
from sklearn.linear_model import Lasso, Ridge
lasso = Lasso(alpha=0.1) # L1
ridge = Ridge(alpha=1.0) # L2Normalises the inputs to each layer during training — zero mean, unit variance within each mini-batch. Reduces internal covariate shift (changing input distributions as earlier layers update).
Benefits: faster training, allows higher learning rates, acts as mild regularisation. Standard in CNNs and feedforward networks. Layer Normalisation is preferred in transformers (normalises across features, not batch).
import torch.nn as nn
layer = nn.Sequential(
nn.Linear(256, 128),
nn.BatchNorm1d(128),
nn.ReLU()
)MLOps applies DevOps principles to ML: reproducible training, automated deployment, monitoring, and retraining.
Key practices:
Linear regression predicts a continuous number. It fits y = wx + b by minimising mean squared error; output is unbounded.
Logistic regression predicts a probability for a class. It passes the same linear combination through a sigmoid to squash it into (0, 1), and trains with log loss (cross-entropy) rather than MSE.
Despite the name, logistic regression is a classifier. It's linear in the sense that the decision boundary is a straight line/hyperplane — which is exactly its limitation on non-linearly-separable data.
Both remain strong baselines: fast, interpretable coefficients, and hard to beat on small tabular datasets with mostly linear signal.
The tree greedily picks, at each node, the feature and threshold that best separates the data — measured by Gini impurity or entropy (information gain) for classification, variance reduction for regression — and recurses until a stopping rule fires.
Strengths: no scaling needed, handles mixed numeric/categorical data, captures non-linear interactions, and the path to a prediction is human-readable.
Weaknesses: a single deep tree overfits badly and is unstable — small data changes give a very different tree. It also produces axis-aligned, staircase boundaries, and biases splits toward high-cardinality features.
Controlled by max_depth, min_samples_leaf, min_samples_split and pruning — but the real fix for the variance is ensembling.
Both are tree ensembles; they attack opposite ends of the bias-variance decomposition.
Rule of thumb: Random Forest for a fast, safe baseline; gradient boosting when you want the best tabular score and can afford to tune.
Needed by: anything distance- or gradient-based — KNN, SVM, K-means, PCA, logistic/linear regression with regularisation, and neural networks.
Not needed by: tree-based models (trees split on thresholds, so monotonic rescaling changes nothing).
Critical detail: fit the scaler on the training set only, then apply it to validation and test. Fitting on the full dataset leaks test statistics into training.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
# scaler is fit inside each CV fold -> no leakage
pipe = Pipeline([
("scale", StandardScaler()),
("clf", LogisticRegression()),
])
pipe.fit(X_train, y_train)Also handle the unseen category at inference — every scheme needs a defined fallback, or production breaks on the first new value.
First ask why it's missing — the mechanism decides the fix:
Techniques: drop (only if few rows or a mostly-empty column), mean/median/mode (simple, shrinks variance), forward-fill for time series, KNN or iterative imputation (MICE), or let the model handle it natively — LightGBM and XGBoost learn a default direction for missing values.
Almost always worth adding an explicit was_missing indicator column: it preserves the signal that imputation destroys.
As the number of features grows, the volume of the space grows exponentially, so your fixed number of samples becomes vanishingly sparse in it. Consequences:
Countermeasures: feature selection, dimensionality reduction (PCA, UMAP), regularisation (L1 drives features to zero), domain-driven feature engineering, and using models that cope better with many features (regularised linear models, tree ensembles).
Note embeddings are the constructive flip side: they map sparse high-dimensional data into a dense low-dimensional space where distance is meaningful again.
Principal Component Analysis finds the orthogonal directions of maximum variance in the data and re-expresses it in those axes. Keeping the top k components gives a lower-dimensional representation that preserves as much variance as possible.
Use it for: compressing correlated features, removing multicollinearity before a linear model, speeding up training, and 2-D visualisation of high-dimensional data.
Costs: components are linear combinations of all original features, so you lose interpretability; it is unsupervised, so a low-variance direction that happens to be the one predictive of your label can be discarded; and it only captures linear structure.
Requirements: standardize first (PCA follows variance, so unscaled large-unit features dominate), and fit on the training set only.
Pick k centroids, then repeat until stable: assign each point to the nearest centroid, then move each centroid to the mean of its assigned points. It minimises within-cluster variance (inertia).
Limitations:
Alternatives: DBSCAN (density-based — finds arbitrary shapes, chooses its own cluster count, labels outliers as noise), hierarchical clustering (dendrogram, no k up front), GMM (soft assignments, elliptical clusters).
No training phase at all — it stores the dataset. To predict, it finds the k closest training points and takes a majority vote (classification) or mean (regression). This makes it a lazy, non-parametric, instance-based learner.
k controls the bias-variance trade: k=1 gives a jagged, high-variance boundary that memorises noise; large k over-smooths toward the majority class. Odd k avoids ties in binary classification.
Costs: prediction is O(n·d) per query, memory holds the whole dataset, features must be scaled, and it degrades badly in high dimensions.
It stays relevant conceptually because vector search is KNN — a RAG retriever is approximate KNN over embeddings, with an ANN index replacing the brute-force scan.
A Support Vector Machine finds the hyperplane that separates classes with the maximum margin — the widest possible gap. Only the points on the margin (the support vectors) determine the boundary; everything else could be deleted without changing it.
C controls the soft-margin trade-off: high C means few misclassifications but a narrow margin (overfit risk); low C accepts errors for a wider, more general margin.
The kernel trick: for non-linearly-separable data, an SVM can operate as if the data were mapped into a much higher-dimensional space — but the algorithm only ever needs inner products, and a kernel function (RBF, polynomial) computes those directly without ever constructing the high-dimensional coordinates. You get a non-linear boundary at linear-ish cost.
Strong on small-to-medium, high-dimensional data (classic text classification). Scales poorly past ~100k rows and gives no calibrated probabilities by default.
It applies Bayes' theorem — P(class | features) ∝ P(class) · P(features | class) — with the "naive" assumption that features are conditionally independent given the class, so the joint likelihood factorises into a simple product.
That assumption is essentially always false (in text, words are highly correlated). It still works because classification only needs the correct class to score highest, not the probabilities to be accurate. The estimated probabilities are usually badly calibrated — over-confident near 0 and 1 — while the ranking survives.
Strengths: trains in one pass, needs little data, handles very high dimensions, excellent baseline for text (spam filtering, topic classification).
Practicalities: use Laplace smoothing so an unseen word doesn't zero out an entire product, and work in log-space to avoid floating-point underflow.
Learning-rate schedule usually matters more than optimizer choice: warmup then cosine decay is the common recipe.
Backpropagation multiplies gradients layer by layer. If the per-layer factors are consistently < 1 the product shrinks toward zero — early layers stop learning (vanishing). If they're consistently > 1 it blows up to NaN (exploding). Depth makes both worse; sigmoid/tanh saturate and make vanishing much worse.
Fixes that made deep networks trainable:
During training, randomly zero each neuron's output with probability p (typically 0.2–0.5). During inference, dropout is switched off and activations are scaled so expected values match (frameworks do this via inverted dropout at train time).
Why it works: no neuron can rely on any specific other neuron being present, so the network can't build fragile co-adapted paths and is forced to learn redundant, distributed representations. It also approximates training an ensemble of exponentially many sub-networks that share weights.
Practical notes: apply it to dense layers rather than convolutional ones (which have their own variants); it interacts awkwardly with BatchNorm, so modern architectures often prefer one or the other; and transformers use it lightly, mostly on attention and residual paths.
A deployed model degrades because the world moved, not because the code changed.
Detection: monitor input feature distributions (PSI, KL divergence, Kolmogorov–Smirnov) and prediction distributions — these are available immediately. Actual accuracy needs ground truth, which often arrives days or months later, so drift metrics are your early warning.
Response: alert thresholds, scheduled or triggered retraining, and always keep a champion/challenger comparison so you can prove the retrained model is actually better before promoting it.
Two cautions: explanations are correlational, not causal, and required by regulation in some domains (credit, hiring) — where an inherently interpretable model may be the safer choice than a post-hoc explanation of a black box.