Chapter 111 — GenAI Interview: Scenario & Design Questions

📖 How to Answer Scenario Questions

Interviewers describe a business problem and ask how you'd build or fix it. Answer in this order:

1. Clarify — users, data, scale, accuracy needs, privacy rules.
2. Approach — RAG / extraction / agent / classification, and why.
3. Architecture — components in order of the request flow.
4. Quality — how you'll evaluate.
5. Risks — security, cost, failure handling.
6. Rollout — pilot first, measure, improve.

1. "Build a chatbot that answers employees' HR policy questions."

• Clarify: number of employees, document sources (PDFs on SharePoint), languages, which policies differ by location/grade.
• Approach: RAG with permission and location filters.
• Flow: SharePoint connector → parse and chunk by policy sections → embeddings + metadata (location, grade, policy name, version) → vector store. Query: rewrite question → hybrid search filtered by the employee's location/grade → rerank → LLM answers with citations → "contact HR" fallback when not found.
• Quality: 150 real HR questions with answers from the HR team; measure correctness and citation accuracy.
• Risks: outdated policies (version metadata, latest only), sensitive questions (salary, harassment → route to HR contacts), PII in logs.
• Rollout: pilot with one department, then company-wide in Teams/Slack.

2. "Extract data from 10,000 invoices per day into our ERP."

• Approach: document AI pipeline with structured output.
• Flow: upload/email → OCR or vision model → LLM extraction into a JSON schema (vendor, invoice number, date, GSTIN, line items, totals) → validation rules in code (totals add up, GSTIN format, date valid, duplicate invoice check) → confidence score → auto-post to ERP if all checks pass, else human review queue.
• Cost: batch processing for non-urgent invoices, cheaper model for simple layouts, cache vendor templates.
• Quality: field-level accuracy on 500 labelled invoices; track human corrections.

3. "Our customer support bot sometimes gives wrong refund information."

1. Collect the wrong conversations; check what was retrieved.
2. Likely causes: outdated or conflicting documents, chunk missing conditions, model guessing.
3. Fixes: clean/version the refund policy docs; get refund eligibility from the order system via a tool (not from documents); stricter prompt with citations and "I don't know"; add these cases to the test set.
4. For actual refunds: rules in code, human approval above a limit.

4. "Summarise long customer-call transcripts for managers."

• Speech-to-text (if audio) → clean transcript → LLM summary with a fixed template (issue, actions taken, sentiment, follow-ups) using structured output.
• Long transcripts: chunk and summarise each part, then combine (map-reduce), or use a long-context model.
• Evaluate with manager feedback and a rubric (missing key points, wrong facts).
• Mask personal data before storing summaries.

5. "Add AI search to our e-commerce site."

• Hybrid search over the product catalogue (title, description, attributes) with filters (price, size, brand).
• LLM to understand queries: "red running shoes under 3000 for women" → structured filters + search text.
• Keep LLM usage light for cost/latency: cache query understanding, use a small model, or use embeddings only.
• Measure click-through rate, conversion, zero-result searches.

6. "Build a text-to-SQL assistant for business users."

• Give the LLM the database schema (table/column descriptions) and examples of good queries.
• Generate SQL → validate: read-only user, only allowed tables, add LIMIT, block dangerous statements → execute → LLM explains the result in plain language and shows the SQL.
• Evaluate on a set of questions with known correct results (compare result sets, not SQL text).
• Risks: wrong joins giving wrong numbers → show the SQL and assumptions; restrict sensitive tables.

7. "The chatbot is too slow — 12 seconds per answer."

Measure each step (retrieval, rerank, LLM time to first token, generation). Typical fixes: stream the answer, use a smaller model for query rewriting, reduce chunks, run retrieval steps in parallel, enable prompt caching, put services in the same region, reduce output length.

8. "Our monthly LLM bill doubled."

Check cost per feature/user from logs: a new feature, longer prompts, more chunks, a loop in an agent, abuse/bots, or a model change. Fixes: rate limits and budgets, caching, routing to smaller models, trimming context, alerts on cost spikes.

9. "A user tricked the bot into revealing its system prompt."

Assume the system prompt is not secret — remove any secrets/internal URLs from it. Add instructions and output filters to reduce leakage, log and monitor such attempts, and ensure the real protections (permissions, tool authorization) are in code.

10. "Client data cannot leave their network."

Options: models hosted in the client's cloud account and region (Azure OpenAI, AWS Bedrock, Google Vertex AI with private networking), or self-hosted open-source models (e.g. Llama/Mistral/Qwen via vLLM) on their GPUs. Trade-off: quality vs control vs operations effort; run the evaluation set on candidate models before choosing.

11. "Generate personalised marketing emails for 1 lakh customers."

Batch processing (batch API) with a template + customer attributes; brand-voice guidelines and examples in the prompt; automatic checks (length, banned words, correct name/offer); sample human review before sending; A/B test against the existing emails; respect consent and unsubscribe preferences.

12. "Build an AI assistant inside our existing MERN application."

• Backend: new Express route /api/assistant with auth, rate limiting, input validation; service layer calling the LLM with RAG/tools; streaming via SSE; conversation stored in MongoDB.
• Frontend: chat component in React, streaming display, citations, feedback buttons; Redux/RTK Query for conversation list.
• Tools call existing APIs with the logged-in user's permissions.
• Logging, evaluation set, feature flag for gradual rollout.

13. "How do you decide if a problem needs GenAI at all?"

Use GenAI for unstructured language tasks (understanding, summarising, generating, extracting from free text). Use normal code/rules for exact calculations, simple lookups and fixed logic; use classic ML for high-volume numeric predictions. If a SQL query or a search box solves it, that's cheaper and more reliable.

← Evaluation, Security & Cost Questions | Index | Next: Hands-On Coding Tasks →