• De-emphasizes coherence with the original text; focuses on factual correctness only.
  • Allows use of general world knowledge as valid ground truth.
  • Only statements that are false, self-contradictory, or implausible count as hallucinations.
  • Minor logical or coherence issues with the original text should not be considered hallucinations.
  • Provides explicit label definitions for “No Hallucination”, “Possible Hallucination”, and “Definite Hallucination”.

V3 Differences from Base:

  • Allows minor invented/unverifiable terms if plausible and not contradicting world knowledge.
  • De-emphasizes off-topic or loosely connected content.
  • Only clear factual errors or implausible claims are considered hallucinations.
  • Plausible but invented terms or creative liberties are treated as “Possible Hallucination” unless they contradict the human continuation or known facts.

V4 Differences from Base:

  • Removes ambiguity around coherence, style, and narrative oddities.
  • Explicitly tolerates creative, loosely grounded content while still catching true factual errors.
  • Instructs not to penalize for minor semantic or logical quirks in story continuations.
  • Narrative oddities, off-topic content, or unusual story events are not hallucinations if plausible or creatively reasonable.

V5 Differences from Base:

  • Most lenient: only clear, unambiguously false, self-contradictory, or impossible statements count as hallucinations.
  • If uncertain, lean toward “No Hallucination”.
  • Unusual, speculative, or imaginative content is not penalized.
  • Plausible inventions or mild factual stretching are at most “Possible Hallucination”.
  • Ignore coherence gaps, logical quirks, or off-topic continuations unless they make the text factually impossible.

A.2.2 Factuality Prompts: without reference

Base Factuality Prompt - without reference

You are a factuality evaluator.

You will be given an original text and a model continuation.

Your task is to determine if the continuation contains any hallucinations, internal inconsistencies, or statements implausible given the original text.

Instructions:
  1. Reason step by step about whether the continuation logically follows from the original text.
  2. At the end, output your reasoning and final label strictly in JSON format.
JSON Output Format:
{
          "reasoning": "your reasoning here",
          "label": "No Hallucination" | "Possible Hallucination" | "Definite Hallucination"
        }