What makes up the cost of a RAG query?

Three pieces: embedding the user's query (cheap), feeding the retrieved chunks plus the query as input to the generation model (often the largest piece), and generating the answer (output tokens). Vector-search compute is usually negligible per query.

Why are retrieved chunks the biggest cost?

Retrieved context is sent as input on every query and can be thousands of tokens. More chunks or larger chunks directly raise the input token bill, so retrieval breadth is the main cost lever.

Is the embedding cost significant?

Usually no. Embedding one short query costs a tiny fraction of a cent. The expensive part is the generation call with the retrieved context attached.

Is anything sent to a server?

No. The full breakdown is computed in your browser. Nothing you enter is uploaded, stored or logged.

Semantic Search Cost Estimator

Semantic search cost estimator

A single RAG (retrieval-augmented generation) query is not one cost — it’s three: embedding the question, sending the retrieved context to the model, and generating the answer. This tool breaks them out so you can see where the money actually goes and project the monthly bill for your query volume.

How it works

The estimator prices each stage and sums them:

embed_cost = (query_tokens / 1e6) × embedding_price
input_cost = ((retrieved_chunks × chunk_tokens) + query_tokens) / 1e6 × gen_input_price
output_cost = (answer_tokens / 1e6) × gen_output_price
per_query = embed_cost + input_cost + output_cost

In almost every real pipeline the input cost dominates because the retrieved context is large and is re-sent on every query. Embedding the query is a rounding error by comparison; vector-search compute is typically negligible per request.

Tips

The biggest savings come from retrieving fewer, better chunks — re-rank and send the top 3 instead of the top 10.
Trim chunk size or summarize retrieved passages before sending them to the generator to cut input tokens.
Use prompt caching for any fixed system prompt or instructions so you only pay full price for the variable retrieved context.
Route simple lookups to a cheaper generation model; reserve the premium model for queries that genuinely need stronger reasoning.