What is paired testing for AI discrimination?

Paired testing sends the AI two prompts that are identical in every way except one protected attribute, such as a name that signals a different ethnicity. If the outputs differ systematically, the attribute is influencing the decision, which is the signature of discrimination.

What is the four-fifths rule?

The four-fifths (or 80%) rule is a US EEOC threshold. If the selection rate for any group is less than 80% of the rate for the highest group, that is treated as evidence of adverse impact. It is a screening signal, not proof, but it triggers a duty to investigate.

Does this prove my AI is discriminatory?

No. It generates the test cases and tells you the analysis method, but a single run is noisy. You must run each prompt many times, aggregate outcome rates per group, and treat any disparity as a prompt to investigate the model, prompt, or training data.

Which laws make this testing relevant?

In the EU, the AI Act classifies hiring and credit AI as high-risk and requires bias testing. In the US, Title VII, the ECOA, and NYC Local Law 144 all create exposure for biased automated decisions. Documented testing is part of a defensible compliance posture.

Does my prompt data leave my browser?

No. The builder assembles every test prompt locally in your browser and outputs plain text. Nothing is uploaded — you run the generated harness against your own model.

AI Discrimination Test Builder

AI discrimination test builder

If an AI system makes or informs a decision about a person — who gets hired, what price they pay, whether their content is removed — you have a legal and ethical duty to check that it is not treating protected groups differently. The cleanest way to detect that is paired testing: send the model two requests that are identical in every respect except a single protected attribute, then see whether the outputs diverge. This tool builds those matched prompt sets for you.

How it works

You paste your real decision prompt and pick the protected characteristics you want to probe — gender, race, age, disability, religion, pregnancy, or name-based proxy signals. For each characteristic the builder produces a set of otherwise-identical variant prompts that change only that one attribute and explicitly hold all other qualifications constant. It then assembles a runnable test harness with the recommended method: run each prompt at least twenty times, record the outcome, compute the positive-outcome rate per group, and apply the four-fifths rule to flag adverse impact.

Tips and notes

Test name proxies, not just explicit attributes. Models often infer ethnicity or gender from a name even when you never state it — that is where real-world bias hides.
Volume matters. A single generation is noise. Aggregate over many runs at a fixed temperature so you are measuring the model, not luck.
A passing test is not a clean bill of health. Paired testing catches first-order disparities; it cannot catch intersectional or context-dependent bias. Treat it as one layer of a broader fairness audit.