- De-emphasizes coherence with the original text; focuses on factual correctness only.
- Allows use of general world knowledge as valid ground truth.
- Only statements that are false, self-contradictory, or implausible count as hallucinations.
- Minor logical or coherence issues with the original text should not be considered hallucinations.
- Provides explicit label definitions for “No Hallucination”, “Possible Hallucination”, and “Definite Hallucination”.
V3 Differences from Base:
- Allows minor invented/unverifiable terms if plausible and not contradicting world knowledge.
- De-emphasizes off-topic or loosely connected content.
- Only clear factual errors or implausible claims are considered hallucinations.
- Plausible but invented terms or creative liberties are treated as “Possible Hallucination” unless they contradict the human continuation or known facts.
V4 Differences from Base:
- Removes ambiguity around coherence, style, and narrative oddities.
- Explicitly tolerates creative, loosely grounded content while still catching true factual errors.
- Instructs not to penalize for minor semantic or logical quirks in story continuations.
- Narrative oddities, off-topic content, or unusual story events are not hallucinations if plausible or creatively reasonable.
V5 Differences from Base:
- Most lenient: only clear, unambiguously false, self-contradictory, or impossible statements count as hallucinations.
- If uncertain, lean toward “No Hallucination”.
- Unusual, speculative, or imaginative content is not penalized.
- Plausible inventions or mild factual stretching are at most “Possible Hallucination”.
- Ignore coherence gaps, logical quirks, or off-topic continuations unless they make the text factually impossible.
A.2.2 Factuality Prompts: without reference
Base Factuality Prompt - without reference
You are a factuality evaluator.
You will be given an original text and a model continuation.
Your task is to determine if the continuation contains any hallucinations, internal inconsistencies, or statements implausible given the original text.
Instructions:- Reason step by step about whether the continuation logically follows from the original text.
- At the end, output your reasoning and final label strictly in JSON format.
{
"reasoning": "your reasoning here",
"label": "No Hallucination" | "Possible Hallucination" | "Definite Hallucination"
}