Chapter 103 — Classic ML System Design (Recommendations, Search, Fraud)

📖 What This Round Tests

Even GenAI teams ask classic ML design: recommenders, ranking, fraud and ads still drive most ML revenue, and LLM features are often plugged into these systems. The interviewer wants a structured path from business problem → data → model → serving → monitoring.

🔍 The 8-Step Framework

1. Clarify the goal — what decision does the model make? Business metric?
2. Frame as ML — classification / regression / ranking; prediction target (label).
3. Data — sources, labels (explicit vs implicit), volume, freshness, biases.
4. Features — user, item, context, interaction features; real-time vs batch.
5. Model — baseline → stronger model; why.
6. Offline evaluation — metrics that correlate with the business goal.
7. Serving — latency budget, architecture (candidate generation → ranking), caching.
8. Online evaluation & monitoring — A/B test, drift, feedback loops, retraining.

🏗️ Design 1 — Recommendation System (e.g. "Products for You" on an e-commerce home page)

Goal: increase purchases (and long-term engagement), not just clicks.

Labels (implicit feedback): impressions → clicks → add-to-cart → purchase. Weight purchases highest; treat "impressed but not clicked" as weak negatives.

Two-stage (or three-stage) architecture

Two-tower model: a user tower (history, profile, context) and an item tower (title, category, price, image embedding) each produce a vector; train so that users' interacted items score high (contrastive with in-batch negatives). Item vectors are pre-computed into an ANN index; the user vector is computed per request.

Features for ranking: user × item affinity (category match), item popularity and price, user's recent activity (session features), context (time, device), cross features, embedding similarity from the two-tower model.

Cold start: new users → popular/trending by region, onboarding preferences, session-based recommendations; new items → content-based embeddings (text/image), exploration slots.

Offline metrics: Recall@K for candidates; NDCG@K, AUC, log-loss for ranking; coverage and diversity. Online metrics: CTR, conversion, revenue per session, long-term retention; guardrails: latency, returns rate.

Pitfalls: feedback loops (model only learns from what it showed — add exploration), position bias (items at the top get more clicks — log position as a feature or use counterfactual methods), popularity bias.

Where LLMs fit: generating item descriptions/embeddings, conversational shopping, explaining recommendations, enriching sparse catalog data.

🏗️ Design 2 — Search Ranking

Goal: users find what they want quickly (purchase or successful click within a session).

Pipeline

1. Query understanding — spell correction, tokenisation, synonyms, intent and category classification, entity extraction ("red nike shoes size 9").
2. Retrieval — inverted index (BM25) + semantic (embedding) retrieval → ~1,000 candidates.
3. Ranking — learning-to-rank (LambdaMART / GBDT, or neural cross-encoder for the top results) using query–document relevance, popularity, personalisation, freshness.
4. Blending — mix result types (products, categories, help articles), business rules.

Learning-to-rank losses: pointwise (predict relevance per doc), pairwise (A better than B — RankNet), listwise (optimise NDCG directly — LambdaMART/LambdaRank).

Labels: human relevance judgments (graded 0–4) for a judged set + click logs (de-biased for position).

Metrics: NDCG@10, MRR, zero-result rate, reformulation rate, time-to-success; online: CTR, conversion, abandonment.

🏗️ Design 3 — Real-Time Payment Fraud Detection

Goal: block fraudulent transactions with minimal friction for good customers. Latency budget: < 100 ms in the payment path.

Labels: chargebacks and confirmed fraud — arrive weeks later (label delay); positives are rare (~0.1%).

Features

• Transaction: amount, merchant category, currency, card-present or not.
• Velocity features: number/amount of transactions for this card/device/IP in the last 1 min / 1 h / 24 h (computed in a streaming system, stored in an online feature store).
• Behavioural: deviation from user's usual amount/location/time.
• Device and network: device fingerprint, IP reputation, distance from the last transaction.
• Graph: shared devices/cards/addresses between accounts (fraud rings).

Model: gradient-boosted trees (fast, strong on tabular data) + rules engine for known patterns; graph features or GNNs for rings.

Decisioning: score → thresholds per segment → approve / step-up authentication (OTP) / manual review / decline. Thresholds tuned on expected cost: fraud loss vs lost revenue and customer friction.

Metrics: recall at a fixed false-positive rate, PR-AUC, fraud loss (₹) prevented, approval rate, manual-review volume.

Challenges: adversarial drift (fraudsters adapt → frequent retraining, monitoring), class imbalance, label delay (use early proxy labels), explainability for disputes (SHAP reason codes), fairness.

🏗️ Design 4 — Content Moderation at Scale (ML + LLM)

Goal: remove policy-violating posts quickly with few false removals.

Cascade:

1. Hash matching for known bad content (very cheap, exact).
2. Fast classifiers (small text/image models) score everything.
3. Uncertain or high-reach items → LLM with the written policy for nuanced judgment and explanation.
4. Human reviewers for appeals and edge cases; their decisions become training data.

Why a cascade: cost — LLM on 100% of traffic is too expensive; small models filter the obvious cases.

Metrics: precision/recall per policy category, time-to-action, appeal overturn rate, reviewer workload.

📊 Quick Reference — Metrics by Problem

🧠 What a Senior Answer Adds

• Starts with the business objective and the cost of errors; picks labels carefully (implicit feedback, delay, bias).
• Uses multi-stage architectures to meet latency at scale.
• Discusses feedback loops, position bias, cold start, drift and how to measure long-term effects.
• Knows when an LLM helps (content understanding, explanations) and when classic models are better (high-QPS tabular scoring).

🎯 Likely Interview Questions

1. Design a recommendation system for an e-commerce home page. — Candidate generation (two-tower + ANN) → ranking (GBDT/deep multi-task) → re-ranking rules; metrics; cold start; feedback loops.
2. Why multi-stage ranking? — Cheap high-recall stage narrows millions to hundreds; expensive accurate model ranks only those within latency.
3. How do you handle cold start? — Popularity/context for new users, content-based embeddings for new items, exploration.
4. What is position bias and how do you correct it? — Higher positions get more clicks regardless of relevance; position features, randomisation, inverse propensity weighting.
5. Pointwise vs pairwise vs listwise ranking? — Per item score vs pair preference vs optimising list metrics like NDCG.
6. Design real-time fraud detection. — Streaming velocity features, GBDT + rules, cost-based thresholds, step-up auth, label delay, adversarial drift.
7. How do you choose fraud thresholds? — Minimise expected cost (fraud loss vs friction/lost sales), per segment, validated online.
8. Offline metric improved but online metric didn't — why? — Offline/online mismatch, leakage, position bias, novelty effects, wrong proxy metric, serving skew.
9. What is a two-tower model? — Separate user and item encoders producing embeddings for efficient ANN retrieval.
10. Where would you use an LLM in a recommender? — Item understanding/embeddings, catalog enrichment, conversational recommendations, explanations — not as the high-QPS ranker.

← GenAI System Design | Index | Next: ML & GenAI Coding Round →