Why developers need a different roadmap
Adding AI to software is not the same skill as training models. As an application developer you rarely touch gradient descent — you call hosted models over HTTP, feed them the right context, and wrap the result in reliable, observable code. The hard parts are not the maths; they are non-determinism, cost, latency, and knowing whether your system actually works. This roadmap orders the work so each layer builds on the last, instead of jumping straight to the flashy parts (agents) before the foundations (evals) are in place.
The roadmap in order
1. LLM APIs. Start by calling a chat completions or messages endpoint directly. Learn system vs. user messages, temperature, max tokens, and streaming. Build a thin wrapper with retries and timeouts so the rest of your app never talks to the raw API. Understand that you are billed per token — see What Is a Token in AI? — and instrument token counts from day one.
2. Structured output and tool use. Make the model return JSON you can parse, and learn function/tool calling so the model can trigger your code. This unlocks classification, extraction, and routing — the bread and butter of real features.
3. Embeddings and RAG. Convert text to vectors, store them, and retrieve the most relevant chunks to ground answers in your own data. Retrieval-augmented generation is how you make a general model speak accurately about your private documents without fine-tuning. Most “hallucination” problems in products are really retrieval problems.
4. Agents. Compose tool calls into multi-step workflows that plan, act, and check results. Agents are powerful but failure-prone, so only build them once you can evaluate single-step calls reliably.
5. Evaluation. Build a labelled test set and score outputs automatically — exact match, rubric, or LLM-as-judge. This is what lets you change a prompt or model and know whether quality went up or down. Skipping it is the number-one reason AI features quietly degrade.
6. Production monitoring. Log inputs, outputs, latency, token usage, and cost per request. Add fallbacks for provider outages, rate-limit handling, and alerts on cost and error spikes.
How to practise without wasting weeks
Build one small but complete project end to end before going deep on any single layer — for example, a support-ticket classifier with a 50-case test set, or a documentation Q&A bot using RAG. A complete vertical slice teaches you more than reading about each technique in isolation, because it forces you to confront cost, evaluation, and error handling together.
When you estimate spend for a feature, use the LLM API Cost Calculator so you size context windows and model choice realistically. Default to the smallest model that passes your evals, cache aggressively, and only escalate to larger models or fine-tuning when measurement — not intuition — tells you to. The developers who succeed with AI are the ones who treat it as ordinary engineering: small, testable changes, measured against a baseline, shipped behind a flag.