How is sensitivity measured?

For each ablation the tool compares the new output against the baseline output using a character-level similarity ratio, then reports the shift as one minus that similarity. A larger shift means removing that phrase changed the output more, so the phrase carries more weight.

Is this a rigorous statistical method?

It is a practical, lightweight ablation, not a formal experiment. Because LLMs are stochastic, scores vary run to run, and a single pass is indicative rather than definitive. Run it a few times or lower temperature for steadier signal.

Does my API key get stored or sent anywhere?

No. The key lives only in the page's memory and is sent directly to OpenAI or Anthropic from your browser. It is never logged, stored, or routed through any Gera server.

How many requests will this make?

One baseline call plus one call per ablated phrase. With the default of four ablations that is five requests. Each uses tokens on your own account, so keep prompts modest while testing.

Prompt Sensitivity Analyzer

Prompt sensitivity analyzer

Two prompts can look almost identical yet behave completely differently because one word is doing all the work. The classic way to find that word is ablation: remove one piece at a time and see how much the result changes. If deleting a phrase barely moves the output, it is decorative; if the output swings wildly, that phrase is load-bearing and worth protecting. This tool automates the ablation loop against your own model using your own API key.

How it works

First the tool sends your complete prompt and records the baseline output. Then it splits your prompt into candidate phrases — sentences and clauses — and, for each one, sends a version of the prompt with that phrase removed. It compares each ablated output to the baseline with a character-level similarity ratio and reports the shift (one minus similarity) as a percentage. Phrases are ranked so the most influential ones rise to the top. Everything runs client side: your key and prompts go straight to OpenAI or Anthropic, never through a server.

Tips and notes

Lower the temperature first. LLMs are stochastic, so a high temperature adds noise that masks the real effect of an ablation. Steadier output makes the sensitivity scores more trustworthy.
Run it more than once. A single pass is indicative. If a phrase scores high across several runs, you can be confident it is genuinely load-bearing.
Watch your token spend. The tool fires one request per phrase plus a baseline, all on your account. Keep test prompts short.
Act on the findings. Tighten or pin the high-shift phrases, and consider trimming the near-zero ones — they are adding length without changing behavior.