Fabrication detection

Fabrication detection is one concept in the safety & determinism thematic island — siblings of safety, scrubbers, determinism, and fingerprint.

applyFabricationGuard rewrites a synthesized turn before it reaches the user. The variable surface — what it removes and reports — is named tool calls the model invented, paragraphs that reference tools that failed or never ran, quantitative content fabricated when most tools failed, and confessions where the model retracts prior work mid-message.

It returns FabricationGuardResult: a triggered boolean, preservedContent, paragraph counts, the unavailableToolNames it acted on, and optional phantomTools, phantomReplacements, ungroundedEntities, dataInflation, and phantomWipeEscalation slots populated only when those signals fire.

Public exports

Public types

applyFabricationGuard has signature (params: ApplyFabricationGuardParams, config: ApplyFabricationGuardConfig) => FabricationGuardResultWithPluginFindings. partitionFailedTools(failedToolNames, errors) returns { apiFailedToolNames, guardBlockedToolNames }; isGuardBlockedFailure(errorMessage) is the underlying predicate.

Result shape

FabricationGuardResult carries the variable surface back to the caller.

FabricationGuardStrategyBundle

The canonical strategy shape is FabricationGuardStrategyBundle from @pleach/core/types/strategies. Hosts populate it once at runtime construction (SessionRuntimeConfig.fabricationGuardStrategy); every consumer site reads from the bundle instead of threading strategies inline. DEFAULT_FABRICATION_GUARD_STRATEGY_BUNDLE exports an empty bundle (quantitativePatterns: []) — tests and hosts without domain patterns use it as-is; the algorithm gracefully skips every optional slot.

The first seven slots feed the lifted applyFabricationGuard algorithm (parameter shape mirrors ApplyFabricationGuardConfig); the five structured-shape detectors at the tail are read directly by graph node consumers and preserve their bag-side return shapes so consumer rewires stay structural-not-semantic.

contributeFabricationGuard plugin hook

A plugin returns a FabricationGuardImpl from contributeFabricationGuard(). The impl is a single object carrying applyFabricationGuard, isGuardBlockedFailure, the per-signal detectors (detectBulkFailureFabrication, detectUnqueriedCitations, detectSelfFabricationConfession, detectMethodResultFabricationConfession, detectIdentifierMismatch, detectCellLineAttributionFabrication, detectMarkdownSmilesMismatch, detectProcessClaimWithoutCompute), and the contentReferencesMissingTools predicate.

Plugin findings emitted by FabricationDetector[] (registered via the sibling contributeFabricationDetectors hook) merge into the returned FabricationGuardResultWithPluginFindings — the substrate's extension of FabricationGuardResult adding a pluginFindings: readonly FabricationFinding[] field. Legacy consumers ignore the new field; H-3.2 consumer rewires read it.

See contributeFabricationGuard for the full method list and registration shape.

Bag-entry retirement (NEAR-READY)

OrchestratorHotpathModules.fabricationGuard is the legacy untyped-bag entry for the same 11-function surface (the surface widened from 1 → 9 functions; today the bag carries 11). The typed FabricationGuardStrategyBundle slot + the contributeFabricationGuard hook supersede it; the entry is in NEAR-READY retirement status. The H-3.3 runtime probe recordFabricationGuardResolverPath emits per-invocation via:"bundle"|"bag-fallback"|"unwired" telemetry so the soak window can close before the bag-fallback path retires.

Two audit gates lock the retirement state:

See /docs/host-adapter for the broader bag-retirement story across the four OrchestratorHotpathModules entries.

Calling the pipeline

import {
  applyFabricationGuard,
} from "@pleach/core/strategies/fabricationDetector";

const result = applyFabricationGuard(
  {
    fullContent: synthesizedTurn,
    failedToolNames: ["search_corpus"],
    emptyResultToolNames: [],
    isResynthesis: false,
    executedToolNames: ["search_corpus", "fetch_url"],
    allKnownToolNames: registry.list(),
    totalToolCount: 2,
    toolResultsRaw: rawToolResultsBlob,
  },
  {
    quantitativePatterns: DOMAIN_QUANTITATIVE_PATTERNS,
  },
);

if (result.triggered) {
  await respondWith(result.preservedContent);
} else {
  await respondWith(synthesizedTurn);
}

The two required arguments are the params envelope and the config. Everything else degrades gracefully.

Running a single detector

The detectors are exported individually so plugin authors and tests can compose them outside the pipeline.

import {
  detectBulkFailureFabrication,
  DEFAULT_BULK_FAILURE_PATTERN_THRESHOLD,
} from "@pleach/core/strategies/fabricationDetectors";

const bulk = detectBulkFailureFabrication({
  content: turnBody,
  failedToolCount: 4,
  totalToolCount: 5,
  quantitativePatterns: DOMAIN_QUANTITATIVE_PATTERNS,
});

if (bulk.detected) {
  console.warn(
    `bulk-failure fabrication: ${bulk.matchedPatterns} pattern hits`,
    bulk.matchedExamples,
  );
}

DEFAULT_BULK_FAILURE_PATTERN_THRESHOLD (3), DEFAULT_BULK_FAILURE_MIN_LENGTH (800), and DEFAULT_BULK_FAILURE_RATIO_THRESHOLD (0.8) ship as exported constants so consumers can tune the gates without re-deriving them.

Contributing a detector through a plugin

The graph's FabricationNode iterates the union of every plugin's contributeFabricationDetectors() return. A detector reads completedTools from the context and returns a FabricationFinding or null.

// lib/plugins/corpusGuard.ts
import type { HarnessPlugin } from "@pleach/core";
import type { FabricationDetector } from "@pleach/core/plugins";

const emptyResultDetector: FabricationDetector = {
  id: "empty-result-claim",
  detect(ctx) {
    const empty = ctx.completedTools.filter(
      (t) => t.name === "search_corpus" && Array.isArray(t.result) && t.result.length === 0,
    );
    if (empty.length === 0) return null;
    if (!/\bfound\b|\bresults? show\b/i.test(ctx.assistantContent)) return null;
    return {
      detectorId: "empty-result-claim",
      severity: "high",
      reason: "Prose claims results when search_corpus returned an empty array",
      evidence: { emptyToolCalls: empty.map((t) => t.id) },
    };
  },
};

export const corpusGuardPlugin: HarnessPlugin = {
  name: "corpus-guard",
  version: "0.1.0",
  contributeFabricationDetectors: () => [emptyResultDetector],
};

Register the plugin once at runtime construction and every turn runs the detector against its completedTools slice.

Host-supplied tool-argument fabrication detectors

The detectors above run against synthesized prose. A second class of detector runs against tool-call arguments before dispatch — when the model invents an argument value that has no provenance in the turn's prior tool results. The pattern composes a host-supplied verdict, a safetyTier field on the tool definition, and one of two routing destinations the runtime offers out of the box.

Worked example: a host wires a detector that flags HTTP URLs the model invented (no prior tool result returned them) versus URLs that appeared in a resolver tool's output earlier in the same turn. The detector returns a verdict; the runtime decides what to do with it based on the tool's safetyTier.

The pattern in four parts

1. Tool authors tag the tool with safetyTier. Three values: "critical", "standard", "advisory". The field is part of the Tools contract — see that page for the values and the contribution path.

2. The host contributes a detector that returns a verdict. The verdict carries a reason discriminator and the suspect argument. The detector reads the tool name, the args, and the prior tool results from the dispatch context.

3. The verdict promotes through ApprovalDecisionKind. The runtime maps the verdict to a discriminator on the approval-decision enum. Hosts can contribute new discriminators alongside the detector (see Interrupts).

4. Routing splits on safetyTier. Two destinations:

   safetyTier	Destination	User can override?
   "critical"	A hard-halt pipeline stage short-circuits dispatch with _recoverable: false. The operator's pre-committed safety policy.	No
   "standard"	The InterruptApprovalCard surfaces with a ground-truth panel showing the suspect arg, the mismatch reason, and the recovery tools.	Yes — accept, edit, or reject
   "advisory"	Probe-only. The detector still fires for observability; no halt and no card.	N/A — nothing to override

The hard-halt stage

The hard-halt routing registers as a pipeline stage at an explicit slot (today, slot 6d2 between the seed validator and the approval stage). Stages at that position run after the model-recoverable predicates but before the user-facing approval card — so a critical tier never reaches the card and the user is never offered a button that would override the operator's policy.

The stage returns an unrecoverable envelope (_recoverable: false, plus a structured error code) and emits a routing-discriminator probe so dashboards can count hard-halts separately from the broader suspect-fire metric.

Why this composes cleanly

The four parts are independent: a tool author sets safetyTier once on the definition. The host wires a detector once at plugin construction. The runtime owns the routing — there's no per-tool glue code. New tools inherit the routing for free as long as the detector recognizes their arg shape.

The audit invariant the substrate enforces: every tool tagged safetyTier: "critical" must be reachable by the host's detector. A "critical" tool with no detector coverage would silently bypass the hard-halt — the registration is the load-bearing artifact and CI fails when coverage is missing.

Adding a custom detector

The bundled detectors handle the common shapes (hallucinated UUIDs, malformed identifiers, file paths that don't exist). Domain-specific detectors register through the contributeFabricationDetectors plugin hook. The hook returns readonly FabricationDetector[]; each detector is an object with an id, an optional version, and a synchronous detect(ctx) method that returns a FabricationFinding or null.

import type { HarnessPlugin } from "@pleach/core"
import type {
  FabricationDetector,
  FabricationDetectorContext,
  FabricationFinding,
} from "@pleach/core/plugins"

const orderIdDetector: FabricationDetector = {
  id: "order-id-format",
  version: "0.1.0",
  detect(ctx: FabricationDetectorContext): FabricationFinding | null {
    // Order IDs are 8 digits prefixed ORD-. Flag any order-shaped token
    // in the synthesized prose that doesn't match the canonical format.
    const malformed = ctx.assistantContent.match(/\bORD-\d{1,7}\b/g)
    if (!malformed) return null
    return {
      detectorId: "order-id-format",
      severity: "medium",
      reason: "Prose cites an order ID that isn't 8 digits prefixed ORD-",
      evidence: { suspects: malformed },
    }
  },
}

export const myPlugin: HarnessPlugin = {
  name: "domain-detectors",
  version: "0.1.0",
  contributeFabricationDetectors: () => [orderIdDetector],
}

Three rules hold for any custom detector:

• detect must be synchronous. Detectors fire on the hot path — per chunk in the streaming case — so a Promise return would block the stream observer contract.
• The detector is side-effect-free. No telemetry writes, no storage reads, no network calls. Detectors classify; they don't enrich.
• The severity field is required. One of "low", "medium", "high". It reports how serious the finding is and is hoisted into the guard's [FabricationGuard] plugin-detector-fired breadcrumb. It does not by itself trigger the hard-halt stage or the approval card — that routing belongs to the tool-argument verdict mechanism above, which keys on the tool's safetyTier, not on a finding's severity.

A related hook — contributeFabricationDetectorRules — lets a plugin contribute per-detector configuration (regexes, allowlists, thresholds) without re-implementing the detector itself. See Plugin contract for the full surface.

Where to go next