Chapter 92 — ML Fundamentals for Senior Interviews

📖 Definition

Machine learning is building a function f(x) → y from examples instead of hand-written rules. A model has parameters learned from data by minimising a loss (how wrong the predictions are) on a training set, and we judge it on data it has not seen.

Senior GenAI interviews still start here: if you can't explain overfitting, leakage or the right metric, nobody trusts your LLM answers.

🔍 The Three Types of Learning

Regression predicts a number; classification predicts a category.

🔍 Bias–Variance Trade-off (in Plain Words)

Total error ≈ bias² + variance + irreducible noise.

Modern twist (senior): very large neural networks show double descent — test error can go down again as you grow the model past the point where it fits the training data perfectly. That's one reason huge, over-parameterised LLMs still generalise.

🔍 Train / Validation / Test Splits

• K-fold cross-validation: split into k parts, train on k−1, validate on 1, rotate; average the scores. Good for small data.
• Time series: never shuffle — train on the past, validate on the future (rolling / forward-chaining splits).
• Group split: keep all rows of one user/patient in the same split, or you leak identity.

⚠️ Data Leakage (The #1 Silent Killer)

Leakage = information in training that won't exist at prediction time. Results look amazing offline and fail in production.

💻 Code Example — A Leak-Free Pipeline (scikit-learn)

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score

numeric = ["age", "income", "tenure_days"]
categorical = ["plan", "country"]

preprocess = ColumnTransformer([
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")),
                      ("scale", StandardScaler())]), numeric),
    ("cat", OneHotEncoder(handle_unknown="ignore"), categorical),
])

model = Pipeline([
    ("prep", preprocess),                        # fitted inside each fold → no leakage
    ("clf", LogisticRegression(C=1.0, class_weight="balanced", max_iter=1000)),
])

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)   # keeps class ratio per fold
scores = cross_val_score(model, X, y, cv=cv, scoring="average_precision")
print(scores.mean(), scores.std())

🔍 Regularisation

📊 Choosing the Right Metric

Precision = TP / (TP + FP)      "Of what I flagged, how much was right?"
Recall    = TP / (TP + FN)      "Of all real positives, how many did I catch?"
F1        = 2 · P · R / (P + R)

🔍 Class Imbalance

1. Use the right metric (PR-AUC, recall@precision).
2. Class weights in the loss (class_weight="balanced").
3. Resampling: undersample majority, oversample minority (SMOTE) — only on the training fold.
4. Tune the decision threshold on validation instead of using 0.5.
5. Focal loss (down-weights easy examples) for deep models.

📊 Classic Models — When to Use What

Bagging = train many models in parallel on bootstrap samples and average (reduces variance). Boosting = train models sequentially, each fixing the previous errors (reduces bias).

Senior point: for tabular business data, gradient-boosted trees usually beat deep learning and LLMs. Saying "I'd start with LightGBM and a logistic-regression baseline" is a strong answer.

🔍 Feature Engineering Essentials

• Numeric: scaling (needed for linear/NN/k-NN, not for trees), log transform for skewed values, binning.
• Categorical: one-hot (low cardinality), target encoding (high cardinality, do it inside CV folds), embeddings.
• Time: hour/day-of-week, time since last event, rolling counts/averages (computed only from the past).
• Text: TF-IDF baseline, or embeddings from a pretrained model.
• Missing values: impute + add an "is_missing" flag (missingness itself can be signal).

🔍 Interpretability

🧠 What a Senior Answer Adds

• Starts from the business goal and cost of errors, then picks the metric.
• Builds a simple baseline first and shows the lift of anything complex.
• Talks about leakage, splits by time/group, and data quality before models.
• Thinks about monitoring and retraining (Chapter on MLOps) — the model is not done at deploy.

🌍 Real-World Impact

• A churn model with 0.95 AUC offline dropped to 0.62 in production because a "cancellation_reason" feature leaked the label.
• Switching a fraud model's metric from accuracy to recall at 95% precision changed which model the team shipped.

🎯 Likely Interview Questions

1. Explain the bias-variance trade-off. — Simple models underfit (high bias), complex ones overfit (high variance); pick complexity and regularisation by validation error.
2. How do you detect overfitting? — Training error much lower than validation error; learning curves diverge.
3. What is data leakage? Give examples. — Info at train time not available at prediction time: target leakage, preprocessing on full data, random split on time series, duplicates.
4. Why is accuracy bad for imbalanced data? — Predicting the majority class scores high while catching nothing; use PR-AUC, recall/precision.
5. ROC-AUC vs PR-AUC? — ROC-AUC is insensitive to class balance and can look great on rare-event problems; PR-AUC focuses on the positive class.
6. L1 vs L2 regularisation? — L1 creates sparse weights (feature selection); L2 shrinks all weights smoothly.
7. Bagging vs boosting? — Parallel averaging to reduce variance vs sequential error-correction to reduce bias.
8. Why do gradient-boosted trees win on tabular data? — Handle mixed feature types, non-linear interactions, missing values, little preprocessing; data is usually too small/structured for deep nets to shine.
9. How do you handle class imbalance? — Right metric, class weights, resampling in train folds only, threshold tuning, focal loss.
10. How do you split time-series data? — Chronologically; train on past, validate on future, rolling windows.
11. What is calibration and why care? — Predicted probabilities match observed frequencies; needed when probabilities drive decisions (pricing, risk). Fix with Platt scaling or isotonic regression.
12. What is double descent? — Test error can fall again when models become much larger than needed to fit the training data.

← Cost & Latency Optimization | Index | Next: Deep Learning & Training →