Retrieval-Augmented Generation injects relevant documents into the model's prompt at query time, so the LLM answers from your private, current knowledge without retraining. It is the standard way to ground an assistant in facts that change or that the model never saw in training.

How does corpus size change the recommendation?

Small corpora can use a lightweight or in-process vector store and simple chunking, while large corpora need a scalable managed vector database, metadata filtering, and often hybrid (keyword + vector) search to keep recall high and latency acceptable.

Which embedding model should I pick?

For most use cases a strong general embedding model (such as a current OpenAI or open-source model) is the right default. Choose larger-dimension models for nuanced semantic recall, smaller/cheaper ones when cost and latency dominate, and a multilingual model if your corpus spans languages.

Why does the planner stress evaluation?

RAG quality lives or dies on retrieval. Without measuring recall and answer faithfulness on a labelled question set, you cannot tell whether a bad answer came from retrieval missing the chunk or the model ignoring it. The planner always recommends a small golden eval set.

RAG Architecture Planner

RAG architecture planner

A Retrieval-Augmented Generation system has four decisions that determine whether it works: how you chunk documents, which embedding model you use, which vector store holds them, and how you evaluate retrieval. Get them wrong and the assistant confidently answers from the wrong context. This planner turns a few facts about your corpus into a coherent recommendation for all four.

How it works

You describe your document types, corpus size, latency target, and budget. The planner applies the standard trade-offs: small corpora get simple fixed-size chunking and a lightweight or in-process vector store; large or mixed corpora get semantic/structure-aware chunking, a scalable managed vector database, metadata filtering, and hybrid keyword-plus-vector search. It recommends an embedding model sized to your quality-versus-cost posture and always includes a retrieval evaluation step — a golden question set scored on recall and faithfulness. The result is a copy-ready stack summary.

Tips and notes

Start with retrieval quality, not the LLM — most “the model is dumb” complaints in RAG are actually retrieval misses. Chunk on natural boundaries (headings, paragraphs) with a small overlap rather than blind fixed windows, and store rich metadata so you can filter before you rank. Add hybrid search early if your queries include exact terms, codes, or names that pure vector search fluffs. Build a small labelled eval set on day one and re-run it on every change. When your gap is behaviour or format rather than knowledge, RAG is the wrong tool — check the fine-tuning decision helper first.