Career roadmap
AI Engineer
Build products on top of foundation models: retrieval, tool use, evaluation, and the plumbing that keeps them cheap and reliable.
Before you start AI Engineer
- Python or TypeScript at working level
- HTTP, REST, JSON
- SQL basics
- Git and one deploy target you have used
Model foundations
Stop treating the model as magic. Know what it charges you and where it breaks.
Everything you send and receive is billed per token, and the context window is a hard ceiling — not a soft suggestion.
Ch 77 — LLM Fundamentals- Tokenisation: why code and non-English text cost more
- Context window vs max output tokens
- What happens when you exceed it (truncation, 400s)
- Counting tokens before a request instead of guessing
The same prompt can return different text. Know which knobs exist and which your model still accepts.
- Temperature and top-p
- Stop sequences
- Why the newest models drop sampling params entirely
- Designing tests around a non-deterministic dependency
Users forgive slow. They do not forgive a blank screen. Streaming is a product requirement, not an optimisation.
Ch 83 — Streaming Responses- Server-sent events end to end
- Backpressure and client disconnects
- Time-to-first-token vs total latency
- Why long outputs need streaming to avoid HTTP timeouts
Ask for a JSON schema, not for good behaviour. This one change deletes a whole class of parsing bugs.
- JSON schema via output_config
- Why 'reply only in JSON' pleas fail at scale
- Server-side validation anyway
- Enums to constrain classification labels
Capability, price, and speed are three axes. Picking one model for everything is the most common junior mistake.
Ch 87 — Cost & Latency Optimization- Cost per request, not cost per million tokens
- Routing easy traffic to a cheaper model
- Batch APIs for non-interactive work
- Budget alerts before the invoice
BuildA streaming CLI chat tool that prints tokens used and cost after every turn.
Prompting & evaluation
Make prompt changes measurable. This is the stage that gets you hired.
Role, task, context, constraints, request — in that order, so the stable part can be cached.
Ch 78 — Prompt Engineering- System prompt vs user message responsibilities
- Constraints and what to do when information is missing
- Saying it once, at normal volume
- XML tags to separate instructions from data
Use examples when the format is easier to show than to describe. The model copies their length and shape, so choose deliberately.
- Picking examples that cover edge cases
- Example count vs token cost
- Why examples beat adjectives for tone
A fixed set of labelled cases plus a rubric a second person could apply. Without this, every prompt opinion is a vibe.
Ch 85 — Evaluation & Hallucination- 50-200 cases, including the failures you have seen
- Writing a rubric that two reviewers agree on
- Freezing the set so results stay comparable
- Knowing when a small-sample win is noise
A model can score outputs at scale, but the judge itself needs auditing against human labels.
- Pairwise comparison over absolute scoring
- Human spot checks on the judge
- Position and verbosity bias
Prompts are code. A prompt edit that ships without a test run is an unreviewed deploy.
- Blocking a merge on an eval drop
- Tolerances: absolute gates block everything, none block nothing
- Versioning prompts so a regression is attributable
BuildAn eval harness that scores three prompt versions across 50 labelled cases and prints a comparison table.
Retrieval (RAG)
Ground answers in real documents, and prove the grounding works.
Chunk size decides retrieval quality more than the model does. Most bad RAG is bad chunking.
Ch 80 — RAG Pipeline- Fixed-size vs semantic vs structural chunking
- Overlap and why it costs you
- Keeping headings and tables intact
- Metadata on every chunk
Text becomes vectors; similarity becomes a distance query.
Ch 79 — Embeddings & Vector Search- Embedding models and dimensions
- Cosine vs dot product
- ANN indexes: HNSW, IVF
- Re-embedding cost when you switch models
pgvector if you already run Postgres. A dedicated store when scale demands it.
- Postgres + pgvector
- Qdrant, Milvus, managed options
- Filtering by metadata alongside vector search
- Index build time and memory
Vector search misses exact terms — codes, names, SKUs. Keyword search catches them. Use both, then rerank.
- BM25 alongside vectors
- Reciprocal rank fusion
- Cross-encoder rerankers
- When reranking is worth the latency
An answer without a source is a guess. A system that cannot say 'not in the documents' will invent.
- Citing chunk ids back to the user
- Explicit refusal when context is missing
- Faithfulness checks
Measure retrieval separately from generation, or you will tune the wrong half of the system.
- Hit rate and recall@k
- MRR
- Per-chunk eval sets
- Failure triage: retrieval or generation?
BuildDocument Q&A over your own PDFs that cites sources and reports retrieval hit rate.
Tools & agents
Let the model act, without letting it act unchecked.
The model does not call your API. It emits a request to call it, and your code decides.
Ch 81 — Tool / Function Calling- Tool schema design and naming
- Parallel tool calls and returning every result
- Strict schemas
- Errors as tool results, not exceptions
A standard way to expose tools and data to models across clients.
Ch 82 — AI Agents & MCP- Server basics
- When MCP beats a bespoke tool
- Auth boundaries
A loop with no stopping condition is an outage with a budget. Design the exit before the entry.
- Step and token caps
- Retries and idempotent tools
- Detecting a stuck loop
- Deciding when an agent is the wrong shape entirely
Anything irreversible — money, email, deletes — goes through a person until the evidence says otherwise.
- Approval queues
- Draft-then-send patterns
- Audit trails
If the model runs code, it runs it somewhere it cannot hurt you.
- Container isolation
- Network egress rules
- Timeouts and resource caps
BuildAn agent that reads a support ticket, queries a real API, and drafts a reply for human approval.
Ship & operate
Run it in production for strangers, not in a notebook for yourself.
Caching is a prefix match. One volatile byte early in the prompt throws away the whole saving.
Ch 87 — Cost & Latency Optimization- Stable prefix, volatile suffix
- Verifying cache hits in usage data
- Silent invalidators: timestamps, unsorted JSON, changing tool lists
- Response caching for repeated questions
Upstream will rate-limit you, time out, and refuse. Plan all three.
- Retry with backoff and a budget
- Fallback models
- Queueing and load shedding
- Graceful degradation in the UI
Log the prompt version, the retrieved chunk ids, the tokens, and the latency. Debugging without them is guesswork.
- Per-request traces
- Cost and latency dashboards
- Sampling traces for eval sets
- Alerting on quality proxies, not just 5xx
Treat every retrieved document and user paste as hostile input that may contain instructions.
Ch 86 — AI Security- Tagging untrusted data
- Authority stays in the system prompt
- Output validation before any action
- Red-teaming your own app
What you log is what you are liable for.
- Redaction before logging
- Retention windows
- Region and data-residency constraints
BuildA deployed service with a dashboard showing p95 latency, cost per request, and error rate.
AI Engineer tools on your CV
- TypeScript or Python
- Anthropic SDK
- NestJS / FastAPI
- Postgres + pgvector or Qdrant
- Promptfoo or a custom eval harness
- Langfuse / OpenTelemetry
- Docker + one cloud runtime
What AI Engineer employers ask to see
- Two deployed apps a stranger can use without you present
- Published eval numbers before and after one change
- A written post-mortem of a failure mode you fixed
The widest-open AI role right now, and the fastest jump from full-stack. Hired by product startups, SaaS companies, agencies, and enterprise innovation teams. No research background needed.
Content last reviewed 2026-08-31. Guidance only — no institute or paid placement is endorsed anywhere in this book.