What are the cost stages in a RAG query?

A retrieval-augmented query has three priced stages: embedding the query (tiny), the generation model reading the query plus retrieved context as input, and the generation model writing the answer as output. Vector search itself is usually a fixed infrastructure cost, not per-token.

Why is retrieved context the biggest input cost?

To ground answers you inject several retrieved chunks into the prompt. Ten chunks of 500 tokens each is 5,000 input tokens before the question is even asked — far more than the query embedding, and it is re-paid on every single query.

Is vector search included?

Vector search is typically billed as flat infrastructure (managed vector DB or self-hosted), not per token, so it is not in the per-query LLM cost here. Add it separately as a fixed monthly line.

How do I cut cost per query?

Retrieve fewer, more relevant chunks (lower top-k), compress or rerank context, cache answers to repeated questions, and use a cheaper generation model for simple lookups.

Knowledge Base Q&A Cost Per Query

What one knowledge base answer really costs

A RAG (retrieval-augmented generation) query feels like one action, but you pay for three stages: embedding the user’s question, the generation model reading the question plus the retrieved context, and the model writing the answer. This calculator prices each stage so you can see exactly where your spend goes and project it across daily volume.

How the per-query cost breaks down

embed_cost  = query_tokens/1e6 × embed_price
gen_in_cost = (query_tokens + context_tokens)/1e6 × gen_in_price
gen_out_cost = answer_tokens/1e6 × gen_out_price
per_query = embed_cost + gen_in_cost + gen_out_cost

The embedding step is almost free — embedding models are cheap and the query is short. The dominant cost is the retrieved context you inject as generation input, because you re-pay for it on every query, and the answer output, which is priced at the higher output rate.

Tips to control RAG spend

Lower your top-k so you inject fewer chunks, rerank to keep only the most relevant ones, and cache answers to repeated questions. For high-volume, low-difficulty lookups, route to a cheap generation model and only escalate hard questions to a frontier model.