How is the efficiency score calculated?

Each model's speed (tokens/sec) and cost (blended USD per 1M tokens) are normalized to 0-1 across the set. The score is: weight × normalized_speed + (1 − weight) × (1 − normalized_cost), so higher is always better and the cost-sensitivity weight controls the trade-off.

Why does throughput matter for cost?

For batch and streaming workloads, slow models tie up connections, raise infrastructure cost, and hurt user experience. A model that is twice as fast for the same token price often wins even if the per-token cost looks identical.

Are the speed numbers exact?

No. Tokens-per-second varies with prompt length, region, load, and whether you stream. The presets are representative editable estimates — benchmark on your own traffic before committing.

What is first-token latency vs throughput?

First-token latency is how long until the model starts responding (matters for interactive UX). Throughput (tokens/sec) is how fast it produces the rest. This tool focuses on throughput-per-dollar but flags latency against your threshold.

Tokens-per-Second Speed vs Cost Calculator

Pick the right model when both speed and cost matter

For streaming chat, real-time agents, and large batch jobs, the cheapest model is not always the best choice — a model that is twice as fast for a similar price clears your queue sooner and frees infrastructure. This tool normalizes tokens-per-second and cost into one efficiency score you can tune with a single cost-sensitivity slider.

How the leaderboard is built

Across the model set, each model’s throughput and cost are scaled to 0-1. The blended score rewards speed and penalizes cost:

score = weight × norm(speed)
      + (1 − weight) × (1 − norm(cost))

At weight = 1 the ranking is pure speed; at weight = 0 it is pure cost; in between you get a balanced view. A latency threshold separately flags models that start responding too slowly for interactive use.

How to use the result

If you are building an interactive assistant, keep the cost-sensitivity weight high and watch the latency flag. For overnight batch jobs where users never wait, push the weight toward cost. Always validate the chosen model’s real throughput on your own prompts and region before standardizing on it.