Build vs buy: the default answer
For an early-stage startup, the default is buy — call a hosted AI API and ship. Foundation models from OpenAI, Anthropic, and Google deliver capability that would take a research team years and millions to approach. You get it behind an HTTP call with no GPUs, no training runs, and no ML hires. “Building” should mean assembling your product around these APIs, not training a model from scratch.
The build option earns its place only when you hit a concrete wall: an unacceptable cost at scale, a quality ceiling the best API cannot clear for your niche, a latency or privacy requirement an external API cannot meet, or proprietary data that genuinely lifts results. Until one of those is measured, building is premature optimisation.
A spectrum, not a binary
“Build vs buy” is really a ladder with rising effort and control:
- Prompting an API — fastest, cheapest, most flexible. Start here for nearly everything.
- RAG (retrieval-augmented generation) — connect the API to your own documents so it answers from your data without retraining. The right step when the gap is knowledge.
- Fine-tuning a hosted model — adjust an existing model on your examples for consistent tone, format, or classification. Worth it only when prompting plateaus.
- Self-hosting an open model (Llama, Mistral, Qwen) — full control and data residency, but you own the infra, scaling, and ops. Justified by privacy, volume, or cost at scale.
- Training from scratch — almost never correct for a startup.
Climb only as far as a real problem forces you.
Cost modeling and lock-in
Model your AI cost as tokens per request × requests × price per token, plus the engineering time to build and maintain each rung of the ladder. At low traffic, API spend is trivial and engineering time dominates — so optimise for speed of iteration, not pennies per call. Plan the levers you will pull at scale: caching repeated answers, routing easy tasks to a smaller cheaper model, and trimming prompts.
On lock-in, LLM APIs are less sticky than they feel. The transport is simple HTTP, so the switching cost lives in provider-specific prompts, tool schemas, and fine-tunes. Keep a thin abstraction layer over your provider, store your own prompts and an evaluation set, and you can re-benchmark a competitor in an afternoon. Deep reliance on one vendor’s proprietary features (assistants, specific tool formats) is where real lock-in accumulates.
A lean startup AI stack
A pragmatic early stack looks like: one primary LLM provider for generation and reasoning, a cheaper small model for high-volume simple tasks, a vector database (pgvector, Pinecone, or Qdrant) if you need RAG, and an observability layer to track cost, latency, and output quality. Add embeddings and a reranker only when retrieval quality demands them. Resist adding tools you cannot yet measure the value of — every dependency is future maintenance.
The winning move for most teams is to buy capability, build the product and data moat around it, and reserve building-from-scratch for the rare, measured case where buying genuinely cannot deliver.