Should I use FastAPI or NestJS for an LLM API?

Both are excellent and the architecture is identical — validate, call the model, stream, protect, document. FastAPI is leaner and pairs naturally with Python ML tooling; NestJS gives more structure (modules, DI, decorators) and suits larger TypeScript teams. Pick the language your team is strongest in; the LLM-specific concerns are the same either way.

Why stream the response instead of returning it all at once?

LLM responses can take many seconds to complete, and a user staring at a spinner that whole time is a poor experience. Streaming relays tokens as the model produces them, so output appears almost immediately and feels fast. It also lets clients abort early, which saves cost. Use server-sent events or chunked transfer for the streaming endpoint.

How do I stop one user from running up a huge bill?

Rate-limit per API key — requests per minute and a daily token budget — and reject requests that exceed it before calling the model. Track token usage from the provider's response and store it per key. Without per-key limits, a single abusive or buggy client can generate an enormous bill, since each request costs real money.

Should the LLM API key live on the server or client?

Always on the server. Your provider key must never reach the browser, or anyone can extract it and spend your money. The client authenticates to your API with its own token; your server holds the provider key, applies your rate limits and validation, and forwards the request. This is the entire reason to build an API instead of calling the provider from the frontend.

How do I document the API?

Generate OpenAPI automatically from your types — FastAPI does this from your Pydantic models and route signatures, NestJS from its DTOs and decorators via the Swagger module. You get an interactive docs page and a machine-readable spec for free, so consumers can read the contract and generate clients without you writing docs by hand.

How to Build an AI-Powered REST API

Why wrap an LLM in your own API

Calling a model provider directly from a frontend leaks your key, gives you no control over cost, and couples your app to one vendor. Putting a thin REST API in front of the model fixes all three: the provider key stays server-side, you apply validation and rate limits, and you can swap providers without touching clients. The architecture is small enough to build in an afternoon. This guide covers it in FastAPI or NestJS, and the tool below generates a starter endpoint contract and a curl command to test it.

How the API is structured

Five pieces, in order. Validation — define the request body (prompt plus options like max tokens and temperature) and reject malformed input with Pydantic models or class-validator DTOs before spending a cent. Model call — forward the validated request to the provider, with the provider key read from an environment variable, never hardcoded. Streaming — expose an endpoint that relays tokens to the client as they arrive via server-sent events or chunked transfer, so output appears instantly. Protection — require an API key or token on every request, and rate-limit per key (requests per minute plus a daily token budget) so no single client can run up your bill. Documentation — generate OpenAPI automatically from your types, giving consumers an interactive docs page and a machine-readable contract.

In FastAPI these map to Pydantic models, a StreamingResponse, a dependency for auth, slowapi or a Redis counter for limits, and built-in /docs. In NestJS they map to DTOs, an SSE or streaming controller, a guard, a throttler module, and the Swagger module. Same shape, different syntax.

Tips and pitfalls

The cardinal rule is that the provider key lives only on the server — the reason to build this API at all is to keep that key off the client. Always rate-limit per key before the model call, since each request costs money and one buggy client can be catastrophic. Stream by default; a multi-second wait with no output feels broken. Track token usage from the provider response and store it per key so you can bill or cap. And validate first, fail fast — never pay for a model call on a request you could have rejected for free. Use the builder below to scaffold the endpoint and a test command.