Yes. The key lives only in your browser memory for the duration of the request and is sent directly to the provider over HTTPS. It is never logged, stored, or sent to any Gera server.

Why are the two replies different lengths or styles?

That difference is exactly what you're measuring. The system prompt steers tone, format, and behaviour, so comparing A and B on the same user message isolates the effect of the prompt change.

Which providers are supported?

OpenAI (chat completions) and Anthropic (messages). Pick the provider that matches your key, then select a model from the dropdown.

Does this cost money?

Yes — each run makes two real API calls billed to your own account at your provider's normal token rates. The token usage shown helps you estimate that cost.

System Prompt A/B Tester (BYO-key)

A/B test two system prompts in one click

Prompt engineering is empirical: the only way to know whether a wording change helps is to run both versions on the same input. This tool sends an identical user message with system prompt A and system prompt B to your chosen model, then shows both replies side by side along with latency and token usage — so you can compare quality and cost directly.

How it works

You bring your own OpenAI or Anthropic API key. When you click Run, the tool fires two requests in parallel — same user message, different system prompt — and renders each response in its own column. For OpenAI it calls the chat-completions endpoint; for Anthropic it calls the messages endpoint with the browser-access header. Everything happens client-side; your key never leaves your machine except in the direct HTTPS call to the provider.

Tips

Change one thing at a time between A and B so you can attribute any difference to that change.
Run the same pair a few times — sampling means a single run can be noisy. Lower the temperature for more repeatable comparisons.
Watch the token usage column: a longer system prompt that barely improves quality may not be worth the per-call cost at scale.