Star on GitHub
GenAI & ML (Senior)Senior Rapid-Fire Q&A

Chapter 105 — Senior GenAI / ML Rapid-Fire Q&A & Leadership Round

📖 How to Use This Chapter

Short answers to say in 20–40 seconds, grouped by topic. Each topic has a full chapter earlier in this section. Questions marked ⭐ come up most often. The final part covers the leadership / behavioural questions that decide senior levels.

🧮 ML Fundamentals

# Question Answer
1 ⭐ Bias–variance trade-off? Simple models underfit (bias), complex ones overfit (variance); choose by validation error and regularise.
2 ⭐ What is data leakage? Training on information unavailable at prediction time — target leaks, preprocessing on full data, random splits on time series.
3 Precision vs recall? Of predicted positives, how many are right vs of real positives, how many were caught.
4 ⭐ Metric for 0.1% fraud? PR-AUC / recall at fixed precision, not accuracy.
5 L1 vs L2? L1 sparse (feature selection), L2 small smooth weights.
6 Bagging vs boosting? Parallel averaging reduces variance; sequential error-fixing reduces bias.
7 Why GBDTs for tabular data? Handle mixed types, non-linearity, missing values; strong with modest data.
8 What is calibration? Predicted probabilities match observed frequencies; fix with Platt/isotonic.
9 Cross-validation for time series? Forward-chaining splits — train past, validate future.
10 How to handle class imbalance? Right metric, class weights, resample train folds only, tune threshold.

🧠 Deep Learning

# Question Answer
11 ⭐ Explain backprop. Chain rule from loss back through layers, reusing stored activations.
12 Adam vs AdamW? AdamW decouples weight decay from adaptive updates — proper regularisation.
13 Why warmup? Early adaptive estimates are noisy; ramping LR avoids divergence.
14 LayerNorm vs BatchNorm? Per example across features vs per feature across batch; transformers use LayerNorm/RMSNorm.
15 Vanishing gradients fix? ReLU-family, residuals, normalisation, good init.
16 BF16 vs FP16? BF16 has FP32's range → no loss scaling needed.
17 Gradient checkpointing? Recompute activations in backward to save memory.
18 Loss is NaN — first checks? LR, precision overflow, bad data, clip gradients, overfit a tiny batch.

🔤 Transformers & LLMs

# Question Answer
19 ⭐ Explain self-attention. softmax(QKᵀ/√d)V — each token mixes values weighted by query–key similarity; causal mask in decoders.
20 Why √d scaling? Keeps logits' variance stable so softmax doesn't saturate.
21 Attention complexity? O(T²d) compute, O(T²) memory naive.
22 ⭐ RoPE? Rotates Q/K by position-dependent angles → relative position in dot products; extendable with scaling.
23 GQA? Query heads share fewer K/V heads → smaller KV cache, faster serving.
24 MoE? Router picks top-k expert FFNs per token: large total, small active parameters.
25 Encoder vs decoder? Bidirectional understanding (BERT) vs causal generation (GPT).
26 Tokenisation effects? Cost, multilingual fairness, arithmetic and spelling weaknesses.
27 Temperature vs top-p? Reshape distribution vs truncate to probable mass.
28 Where are most parameters? Feed-forward layers (~2/3).

🏋️ Training & Alignment

# Question Answer
29 ⭐ Stages of building a chat model? Pretrain → SFT → preference tuning (RLHF/DPO) → RL for reasoning + safety.
30 Chinchilla scaling? ~20 tokens per parameter compute-optimal; in practice smaller models trained longer for cheaper inference.
31 Training compute estimate? ≈ 6 × params × tokens FLOPs.
32 ⭐ Memory for full fine-tune? ~16 bytes/param with AdamW mixed precision + activations.
33 ZeRO stages? Shard optimizer states (1), + gradients (2), + parameters (3).
34 Tensor vs pipeline parallel? Split matrices across GPUs vs split layers across GPUs.
35 ⭐ RLHF in three steps? Preference data → reward model → PPO with KL penalty to reference.
36 ⭐ DPO vs PPO? DPO optimises preference pairs directly with a classification loss — no RM or RL loop; simpler and stabler.
37 Reward hacking? Exploiting RM flaws (length, flattery); KL penalty, better RMs, length control.
38 Why mask prompt tokens in SFT? Learn to produce responses, not predict user text.

🎛️ Fine-Tuning

# Question Answer
39 ⭐ Fine-tune vs RAG? Fine-tune for behaviour/format/cost; RAG for knowledge and freshness.
40 ⭐ Explain LoRA. Frozen W + low-rank BA update scaled by α/r; B starts at zero; mergeable.
41 QLoRA? LoRA over a 4-bit NF4 frozen base with double quantisation.
42 Catastrophic forgetting fix? PEFT, low LR, few epochs, mix general data, regression evals.
43 How much data? Hundreds–thousands of high-quality examples for style/format; quality over quantity.

⚡ Inference

# Question Answer
44 ⭐ Prefill vs decode? Prefill compute-bound (whole prompt); decode memory-bandwidth-bound (one token/step).
45 ⭐ KV cache size? 2 × layers × kv_heads × head_dim × bytes per token.
46 Continuous batching? Requests join/leave every decode step → high utilisation.
47 PagedAttention? Paged KV-cache blocks → no fragmentation, bigger batches, prefix sharing.
48 FlashAttention? Tiled, IO-aware exact attention in SRAM; O(T) memory.
49 ⭐ Speculative decoding? Draft proposes, target verifies in one pass; same output distribution, 2–3× faster.
50 GPTQ vs AWQ? Second-order error minimisation vs activation-aware protection of salient weights.
51 TTFT vs TPOT levers? TTFT: queue, prompt length, prefix cache; TPOT: model size, batch, bandwidth, quantisation.
52 GPUs for 70B bf16? 140 GB weights → ≥ 2 × 80 GB plus KV cache headroom.

🔎 Retrieval & RAG

# Question Answer
53 ⭐ Debug a bad RAG answer? Check retrieval first (recall@k); then reranking, prompt, faithfulness.
54 Hybrid search? Dense meaning + sparse exact terms, fused (RRF).
55 Bi- vs cross-encoder? Fast separate embeddings for recall vs joint scoring for precision.
56 HNSW vs IVF-PQ? High-recall memory-heavy graph vs compressed clustered index.
57 Permissions in RAG? ACL metadata filtered at retrieval, never trusted to the LLM.
58 RAG vs long context? Long context for small static corpora with caching; RAG for big, changing, permissioned data.
59 Contrastive embedding training? Positive pairs, in-batch and hard negatives, InfoNCE loss.

📏 Evaluation & Safety

# Question Answer
60 ⭐ How do you evaluate an LLM feature? Golden set with slices, programmatic checks, calibrated judges, humans, red teaming, then online tests.
61 ⭐ LLM-judge biases? Position, verbosity, self-preference; swap order, rubrics, calibrate vs humans.
62 Is a 2% win-rate gain real? Check sample size and confidence intervals; use paired comparisons.
63 Measure hallucination? Claim-level faithfulness vs sources on a labelled set + production sampling.
64 Prompt injection defence? Treat outputs as untrusted, least-privilege tools, authorization in code, human approval.
65 Guardrail metrics? Must-not-regress metrics (latency, cost, safety) during experiments.

🛠️ MLOps / LLMOps

# Question Answer
66 ⭐ Training–serving skew? Feature computed differently offline vs online; shared definitions, feature store, log serving features.
67 Data vs concept drift? Inputs change vs input→output relationship changes.
68 Safe model rollout? Shadow → canary → A/B → rollout with auto-rollback.
69 What to version for LLM apps? Model, prompt, retrieval config, tools, eval set.
70 Self-host vs API? Data control, steady high volume, customisation vs ops burden and idle GPU cost.

🏗️ System Design

# Question Answer
71 ⭐ Agent vs workflow? Workflow for known steps; agent when the path depends on intermediate results.
72 Where to enforce business rules? In tool/service code, never only in the prompt.
73 Cut LLM cost 50%? Routing, prompt caching, shorter outputs, batch API, smaller/distilled models, quantised self-hosting.
74 Recommender architecture? Candidate generation (two-tower + ANN) → ranking → re-ranking rules.
75 Offline up, online flat — why? Metric mismatch, leakage, position bias, skew, novelty effects.

🧭 Leadership & Behavioural Round (Senior / Staff)

Use the STAR structure (Situation, Task, Action, Result) and include numbers. Senior answers show judgment, influence and ownership — not just coding.

Questions to prepare stories for

  1. Tell me about an ML/GenAI system you designed end to end. — Problem, constraints, architecture choices and why, evaluation, results (metric + business impact), what you'd do differently.
  2. A time a model failed in production. — Detection (monitoring), mitigation (rollback), root cause (drift/skew/leak), long-term fix (tests, alerts), what changed in process.
  3. A time you said no to using an LLM (or ML). — Simpler rules/SQL/classic model met the need with lower cost and risk; how you convinced stakeholders with data.
  4. Disagreement with a product manager or senior engineer. — Shared goal, data/experiment to resolve it, outcome, relationship afterwards.
  5. Prioritising under ambiguity. — How you chose between improving quality, cutting cost, or shipping features (impact vs effort, risk).
  6. Mentoring / raising the bar. — Introduced eval standards, code review practices, design docs; how the team improved.
  7. Handling a responsible-AI concern. — Found bias/safety/privacy risk, escalated, changed design or launch plan.
  8. Influencing without authority. — Got another team to adopt your platform/eval framework.
  9. Biggest technical mistake. — Honest, what you learned, how you prevent it now.
  10. How do you keep up with the field? — Papers you read selectively, reproducing results, internal reading groups, separating hype from production value.

Strong-answer checklist

  • Starts with the business problem, not the model.
  • States trade-offs you considered and why you chose one.
  • Shows measurement (offline and online) and honest results, including what didn't work.
  • Mentions people: who you aligned, how you communicated, who you helped grow.
  • Ends with a lesson that changed how you work.

🗣️ "Walk Me Through a GenAI Project" — Answer Template

"At [company], support agents spent [X min] per ticket searching docs. I led a RAG assistant for [N] agents. We indexed [sources] with structure-aware chunking and ACL metadata, used hybrid search with a cross-encoder reranker, and a [model] for cited answers, with a smaller model for query rewriting. Before building, we created a [300]-question golden set with the support leads; recall@10 went from [62%] to [91%] after adding hybrid search and contextual chunk headers, and faithfulness reached [96%] with a calibrated LLM judge. We rolled out via shadow mode and a [10%] canary. Average handle time dropped [28%], and cost stayed at [₹X] per ticket thanks to prompt caching and routing. The biggest lesson: most quality gains came from retrieval and evaluation, not from changing the model."

Replace every bracket with your real numbers — specifics are what senior interviewers remember.

← ML & GenAI Coding Round | Index | Next: GenAI Self-Intro & Project Questions →