Summarization pipeline cost planner
When a document is larger than the model’s context window, you cannot summarize it in one call — you run a map-reduce pipeline: summarize each chunk, then combine the summaries, recursively, until one final summary remains. The cost is easy to underestimate because the merge passes add up. This planner computes the chunk count, the recursive merge passes, and the total cost for the whole run.
How it works
Your document token count divided by the chunk size gives the number of map chunks. Each map call reads a chunk (input) and writes a per-chunk summary (output) at the target summary length. The chunk summaries are then combined in a reduce stage: if they do not all fit in one call, they are merged in groups and re-merged recursively, and the planner counts those merge passes. Everything is priced at the selected model’s input and output rates to produce a total pipeline cost.
Tips and notes
- Bigger chunks, fewer calls. Larger chunks cut the number of map calls and merge passes, but leave room for the summary output within the window.
- Split the model choice. Run the map stage on a cheap model and the final reduce on a stronger one — most of the tokens are in the map stage.
- Pre-filter first. Dropping boilerplate or irrelevant sections before summarizing is the cheapest token you will ever save.
- Estimates only. Real summaries vary in length; use realistic targets and confirm pricing before budgeting a large run.