How does it score without an AI judge?

It applies heuristic rubrics over the text — looking for reasoning markers, explanation length, hedging, and lexical overlap with prior turns as a consistency proxy. It is a fast, deterministic first pass, not a substitute for human or LLM-judge review.

What input format does it accept?

Either a JSON array of objects with role and content fields, or plain text with lines tagged user: and assistant:. The parser detects which one you pasted.

What does the consistency score mean?

It estimates how well each assistant turn stays grounded in the conversation by measuring how much of its content connects to earlier turns. Sudden topic jumps or self-contradiction phrases lower the score.

Can it detect contradictions?

It flags surface-level contradiction cues like reversing earlier statements or saying "actually, no" after a confident claim. It cannot verify factual truth, so deep logical contradictions still need human review.

Is my conversation uploaded anywhere?

No. Parsing and scoring happen entirely in your browser. Nothing leaves the page.

Socratic Dialogue Evaluator

Socratic dialogue evaluator

Multi-turn conversations are where LLMs most often go wrong: a model that answers turn one beautifully can contradict itself by turn four, lose the thread, or give shallower answers as the context fills. The Socratic dialogue evaluator scores each assistant turn in a chat log on reasoning depth, internal consistency, and alignment with earlier turns — using heuristic rubrics that run instantly in your browser, no API key required.

How it works

Paste your conversation as JSON ([{ "role": "...", "content": "..." }]) or as plain text with user: / assistant: tags. The evaluator parses it, then scores each assistant turn across three dimensions:

Reasoning depth — rewards explicit reasoning markers (“because”, “therefore”, “if… then”), structured explanation, and worked steps; penalizes one-line non-answers.
Consistency — flags contradiction cues (“actually, no”, “I was wrong”, “ignore that”) and reversals against what the model said earlier.
Alignment — measures how much each turn connects back to the conversation so far, as a proxy for staying on-thread rather than drifting.

Each turn gets a composite score and notes, and you get a conversation-level average so you can compare runs.

Tips and examples

Use the per-turn view to find the exact point where a conversation degrades — often you’ll see depth fall off a cliff once the context gets long, which is a cue to summarize and restart. When building evaluation datasets, a low consistency score is a good filter for turns worth a human’s attention. Remember the scores are heuristic: they reliably catch shallow, drifting, or self-reversing turns, but they can’t judge whether a claim is factually true — pair the tool with spot-checks for anything high-stakes.