Persona consistency checker
A chatbot that opens warm and on-brand but drifts into generic assistant-speak ten turns later erodes trust fast. This tool gives you a quick, repeatable way to test whether a single model output still matches the persona you defined. You paste the persona system prompt and a sample response, and it flags the passages most likely to have broken character.
How it works
The checker pulls signals straight from your persona prompt. It extracts a tone profile from descriptive adjectives, builds a banned-phrase list from any “never say” or “avoid” instructions, and watches for meta-references — phrases like “as an AI language model” or “I cannot” that shatter an in-world persona. It then scans the output for vocabulary overlap with the persona’s own words, counts banned-phrase and meta hits, and produces a consistency score from 0 to 100 alongside a list of the specific lines that triggered each flag.
Tips and notes
- Be explicit in the persona prompt. The more concrete the “always” and “never” rules, the more the checker has to test against. Vague personas score vaguely.
- Test the tail of long chats. Drift accumulates, so the most useful sample is a late-conversation reply, not the first greeting.
- Read every flag. A high score is reassuring but not proof; the flags are the real value because they point you to exact lines to rewrite.
- Tighten, then re-run. Use the flags to add banned phrases or sharpen tone words in the prompt, then paste a fresh output and confirm the score climbs.