How is the self-hosting cost calculated?

It assumes you rent a GPU instance that runs continuously for the month, so cost equals the hourly rate multiplied by roughly 730 hours. The calculator ignores idle time because a dedicated instance is billed whether or not it is busy.

What does the break-even mean?

It is the monthly request volume at which the API token bill equals the fixed monthly GPU cost. Below it, API is cheaper because you only pay per use; above it, the fixed GPU cost is spread over enough requests to win.

Does self-hosting have hidden costs?

Yes. Beyond the GPU you pay for setup, monitoring, scaling, and engineering time, and you carry reliability risk. The calculator captures the raw compute comparison; weigh the operational burden separately.

Is anything sent to a server?

No. The comparison runs entirely in your browser. You enter only counts and rates, and nothing is uploaded, stored, or logged.

Open-Source vs API Cost Comparison

Open-source vs API cost comparison

Renting an LLM API is pay-as-you-go: cheap at low volume, expensive at scale. Self-hosting an open-source model like Llama 3 flips that — a fixed monthly GPU bill that gets cheaper per request the more you use it. This tool finds the break-even volume where self-hosting starts to win.

How it works

The API side is simple: total tokens (volume × tokens per request, split into input and output) priced at the model’s per-million rates gives a monthly bill that scales linearly with usage.

The self-hosted side is a fixed cost: a dedicated GPU instance billed by the hour, running continuously (~730 hours/month), regardless of how busy it is. Setting the two equal and solving for volume gives the break-even — the monthly request count above which the fixed GPU cost is spread thinly enough to beat per-request API pricing.

Tips and notes

Self-hosting is a fixed cost, not free. A 24/7 GPU costs the same whether you serve ten requests or ten million — utilisation is everything.
Add the operational tax. Setup, autoscaling, monitoring, and on-call carry real engineering cost and risk that this raw compute comparison omits.
Batch and autoscale to improve economics. Bursty traffic wastes a dedicated GPU; serverless GPU or batching can lower the effective self-hosted cost below the always-on assumption used here.