Why does batch size matter for embeddings?

Embedding APIs let you send many inputs in one request. Larger batches mean fewer HTTP round-trips and less per-request overhead, so a big job finishes much faster — until you hit the tokens-per-minute rate limit, which caps how much you can push per window.

Does batching change the dollar cost?

No — you pay per token regardless of how the tokens are grouped. Batching changes job duration and request count, not the token bill. This tool shows both so you optimise speed without surprises on cost.

What batch size should I pick?

The largest batch your provider allows that still keeps each request comfortably inside the TPM limit and within any per-request input cap. The tool highlights the recommended size based on your numbers.

Is my data sent anywhere?

No. All calculations run in your browser. Nothing you enter is uploaded, stored or logged.

Batch Size Optimizer for Embedding Jobs

Optimize embedding batch size for speed and cost

Embedding a large corpus is throughput-bound: the token bill is fixed, but how fast the job finishes depends on batch size. Bigger batches cut per-request overhead and finish sooner — until you hit the model’s tokens-per-minute (TPM) rate limit. This tool sweeps batch sizes for your corpus and shows job duration, request count and cost so you can pick the fastest size that stays inside the limit.

How it works

For a corpus of N documents at t tokens each, a batch of b documents needs:

total_requests = ceil(N / b)
tokens_per_request = b × t

Job time is bounded by two things — request latency (total_requests × latency, reducible with parallelism) and the TPM ceiling (total_tokens / TPM minutes, which no batching can beat). The tool reports the larger of the two as the realistic floor, and flags batch sizes whose per-request token count would exceed the TPM window or a typical input cap.

Tips for fast embedding jobs

Max out the batch within limits. Most embedding endpoints accept dozens to thousands of inputs per call — use the largest the API allows.
Run batches in parallel up to TPM. Once the batch is large, concurrency fills the remaining TPM headroom; beyond that you only earn 429s.
Pre-dedupe and chunk well. Embedding identical or near-identical chunks wastes tokens; clean the corpus before the job.
Use the batch API for non-urgent jobs. Asynchronous batch endpoints are often cheaper and dodge real-time rate limits entirely.