Chapter 109 — GenAI Interview: Agents, Tool Calling & Frameworks

📖 How to Use This Chapter

Questions on function/tool calling, AI agents, MCP and frameworks like LangChain and LangGraph. For more depth, see AI / LLM Engineering: Tool Calling and AI Agents & MCP.

1. What is tool (function) calling?

You describe functions (name, description, JSON parameters) to the LLM. When the model decides a tool is needed, it returns a structured request (get_order_status({ orderId: "A123" })) instead of text. Your code runs the function and sends the result back; the model then writes the final answer. The model never executes anything itself.

2. Show a tool-calling loop.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const tools: Anthropic.Tool[] = [
  {
    name: "get_order_status",
    description: "Get the current status of a customer's order by order ID.",
    input_schema: {
      type: "object",
      properties: { orderId: { type: "string", description: "Order ID like A123" } },
      required: ["orderId"],
    },
  },
];

async function runTool(name: string, input: any, userId: string) {
  if (name === "get_order_status") {
    // authorization in CODE: only this user's orders
    const order = await Order.findOne({ orderNo: input.orderId, userId }).lean();
    return order ? { status: order.status, eta: order.eta } : { error: "Order not found" };
  }
  return { error: `Unknown tool ${name}` };
}

export async function chat(userId: string, userMessage: string) {
  const messages: Anthropic.MessageParam[] = [{ role: "user", content: userMessage }];

  for (let step = 0; step < 5; step++) {                    // hard limit on loops
    const res = await client.messages.create({
      model: "claude-sonnet-5-5",
      max_tokens: 1024,
      tools,
      messages,
    });
    if (res.stop_reason !== "tool_use") {
      return res.content.find((b) => b.type === "text")?.text ?? "";
    }
    messages.push({ role: "assistant", content: res.content });
    const results = await Promise.all(
      res.content
        .filter((b): b is Anthropic.ToolUseBlock => b.type === "tool_use")
        .map(async (b) => ({
          type: "tool_result" as const,
          tool_use_id: b.id,
          content: JSON.stringify(await runTool(b.name, b.input, userId)),
        })),
    );
    messages.push({ role: "user", content: results });
  }
  return "Sorry, I couldn't complete that request.";
}

3. What is an AI agent?

An LLM that works in a loop: decides the next step, calls tools, looks at results, and repeats until the goal is done. Example: "Find all invoices over ₹50,000 from last month and email a summary to finance" — search, filter, summarise, send.

4. Agent vs workflow — when do you use each?

• Workflow: steps are known in advance (extract → validate → save). Predictable, cheaper, easier to test.
• Agent: steps depend on what is found along the way (research, troubleshooting). Start with a workflow; use an agent only when needed.

5. How do you make agents safe?

• Least-privilege tools (read-only where possible).
• Authorization and business limits in code (refund only if amount ≤ limit and order belongs to user).
• Human approval for risky actions (payments, deletes, sending emails).
• Max step limits and timeouts.
• Validate tool inputs; idempotency keys for actions.
• Audit log of every tool call.
• Treat tool outputs and documents as untrusted (prompt injection).

6. What is MCP (Model Context Protocol)?

An open standard for connecting AI applications to tools and data sources. A service exposes its tools/resources once as an MCP server, and any MCP-compatible client (Claude, IDEs, agent frameworks) can use them — instead of writing custom integrations for each app.

7. LangChain vs LangGraph vs LlamaIndex?

Interview tip: show you know the trade-off — frameworks speed up prototypes, but add abstraction; many teams use the SDK directly for core production paths.

8. How do agents remember things?

• Short-term: conversation history in the prompt (summarise or trim when long).
• Long-term: store facts/preferences in a database or vector store and retrieve when relevant.
• State: workflow state saved in a DB (LangGraph checkpoints) so long tasks can resume.

9. What is a multi-agent system? Is it always better?

Several specialised agents (researcher, writer, reviewer) or an orchestrator delegating to sub-agents. Useful for big tasks that can be split in parallel, but costs more tokens and is harder to debug — use only when a single agent with good tools isn't enough.

10. How do you handle tool errors?

Return clear error messages to the model as tool results ("Order not found", "Rate limit, try later"), retry transient errors with backoff in code, and stop after a limit with a helpful message to the user.

11. How do you test an agent?

Scenario test set (input → expected tool calls and final outcome), mock tools for deterministic tests, check that forbidden actions are never called, measure task success rate, steps taken, cost and latency, and review traces (LangSmith, Langfuse, or your own logs).

12. How do you stream agent progress to the UI?

Server-Sent Events or WebSockets: send events like thinking, tool_call: searching orders…, tool_result, then streamed answer tokens — users see progress instead of waiting.

13. What is "human-in-the-loop"?

The agent pauses before a sensitive action and asks a person to approve or edit — e.g. shows a drafted email or a refund for approval. Implemented by saving state, notifying the user, and resuming after approval.

14. How do you control agent cost?

Step limits, smaller models for simple steps (routing, classification), caching tool results, trimming conversation history, prompt caching, and monitoring cost per task.

15. Scenario — "Build an agent that books meetings."

Tools: check_calendar(user, dateRange), find_free_slots(attendees, duration), create_event(...) (requires confirmation). Flow: understand request → check availability → propose slots → ask user to confirm → create event → send confirmation. Guardrails: only the user's own calendar, time-zone handling, no double booking (check again just before creating).

← RAG & Vector Database Questions | Index | Next: Evaluation, Security & Cost Questions →