Providers

A provider is what the runtime invokes to talk to a model. @pleach/core ships two — AiSdkProvider (Vercel AI SDK) and AnthropicSdkProvider (Anthropic's official SDK) — plus one interface (AgentProvider) you implement to wrap any other SDK.

The underlying SDK packages are optional peer deps. The runtime imports them dynamically at provider construction, so you only install the ones you use. See Model resolution matrix for how (family × callClass) resolves to a concrete model id, and Tools for how ProviderToolDef is built.

This page is the reference for the routing cluster's transport layer. The three concepts that decide what fires through which provider — CallClass, Seam, family-lock — are framed in Family-lock → the routing cluster.

ProviderFamily is a closed union of nine: anthropic | openai | google | deepseek | moonshot | mistral | qwen | thinkingmachines | xai. Each family locks the tokenizer, prompt-cache key, tool-call dialect, and refusal pattern — see Family-lock. The transport is one of four: native | openrouter | byok-native | byok-openrouter, locked at session start and never silently mutated.

Same shape, different provider

The fastest swap path is AiSdkProvider + OpenRouter. The model identifier is a <family>/<model> string — change it, change providers. Everything else on the page stays identical.

// Anthropic
import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { AiSdkProvider, SessionRuntime } from "@pleach/core";

const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! });

const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: openrouter("anthropic/claude-sonnet-4-5"),
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

// OpenAI — same file, change one string
const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: openrouter("openai/gpt-4o"),
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

// Google Gemini — same file, change one string
const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: openrouter("google/gemini-2.5-flash"),
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

One OPENROUTER_API_KEY, nine families (anthropic | openai | google | deepseek | moonshot | mistral | qwen | thinkingmachines | xai), identical request shape. The runtime's family-lock reads the <family>/ prefix and locks the tokenizer, prompt-cache key, and tool-call dialect for the session — no silent cross-family fallback. Want direct vendor SDKs instead? Drop the OpenRouter wrapper for @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google etc. — covered below.

import { AiSdkProvider, AnthropicSdkProvider } from "@pleach/core";
import type {
  AgentProvider,
  AgentExecutionConfig,
  ProviderMessage,
  ProviderToolDef,
  ProviderCapabilities,
  ProviderStreamEvent,
  TokenUsage,
} from "@pleach/core/providers";

AiSdkProvider (Vercel AI SDK)

Wraps streamText from ai@6.x. Right pick when you want unified provider switching, the AI SDK's tool dialect, and the ecosystem of community providers. The recommended default — pair with OpenRouter to reach any family from one key.

import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { AiSdkProvider, SessionRuntime } from "@pleach/core";

const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! });

const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model:    openrouter("anthropic/claude-sonnet-4-5"),
    maxSteps: 5,
  }),
  storage:  supabaseAdapter,
  userId:   "user_123",
});

Drop the OpenRouter wrapper for a direct provider package any time — @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, etc.:

import { anthropic } from "@ai-sdk/anthropic";

const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: anthropic("claude-sonnet-4-5"),
    maxSteps: 5,
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

Config

Install the ai package plus a provider package — OpenRouter is the default; switch to @ai-sdk/anthropic / @ai-sdk/openai / @ai-sdk/google for direct vendor access:

npm install ai @openrouter/ai-sdk-provider   # or @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, etc.

AnthropicSdkProvider

Wraps @anthropic-ai/sdk directly. Right pick when you want native Anthropic features (prompt caching, extended thinking, tool-use beta flags) without going through a unified wrapper.

import { AnthropicSdkProvider, SessionRuntime } from "@pleach/core";

const runtime = new SessionRuntime({
  provider: new AnthropicSdkProvider({
    apiKey:    process.env.ANTHROPIC_API_KEY!,
    model:     "claude-sonnet-4-5",
    maxTokens: 4096,
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

Config

npm install @anthropic-ai/sdk

The AgentProvider interface

Implement this to wrap any other SDK — OpenAI directly, a custom gateway, a local Ollama instance, anything that streams.

interface AgentProvider {
  execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent>;
  abort(): void;
  readonly capabilities: ProviderCapabilities;
}

AgentExecutionConfig

What the runtime hands you per call.

ProviderMessage

The normalized message shape your provider converts to its native format.

interface ProviderMessage {
  id?: string;
  role: "user" | "assistant" | "system" | "tool";
  content: unknown;
  tool_calls?: Array<{ id: string; name: string; arguments: unknown }>;
  tool_call_id?: string;
  name?: string;
}

ProviderStreamEvent

A tagged union. The runtime adapts these into the public StreamEvent consumers see.

type ProviderStreamEvent =
  | { type: "message.start";    messageId: string; role: string }
  | { type: "message.delta";    delta: string }
  | { type: "message.complete"; messageId: string; usage?: TokenUsage }
  | { type: "thinking.delta";   delta: string }
  | { type: "thinking.complete" }
  | { type: "tool.started";     toolCallId: string; toolName: string; arguments?: unknown }
  | { type: "tool.completed";   toolCallId: string; toolName: string; result?: unknown }
  | { type: "tool.failed";      toolCallId: string; toolName: string; error: string }
  | { type: "error";            error: string; code?: string }
  | { type: "done" };

TokenUsage carries inputTokens, outputTokens, totalTokens, cacheReadTokens, cacheWriteTokens — all optional, all surfaced on message.complete.

ProviderCapabilities

Feature detection so the runtime knows what to ask for.

interface ProviderCapabilities {
  streaming:         boolean;
  toolCalling:       boolean;
  thinking:          boolean;
  structuredOutput:  boolean;
  maxContextTokens:  number;
  parallelToolCalls: boolean;
}

The runtime reads capabilities to decide what to ask for. If toolCalling is false, tool-using sessions won't dispatch tools through this provider; if thinking is false, thinking-delta events are dropped.

Minimal custom provider sketch

// lib/providers/myProvider.ts
import type { AgentProvider, AgentExecutionConfig, ProviderStreamEvent } from "@pleach/core";

export class MyProvider implements AgentProvider {
  readonly capabilities = {
    streaming:         true,
    toolCalling:       true,
    thinking:          false,
    structuredOutput:  false,
    maxContextTokens:  128_000,
    parallelToolCalls: false,
  };

  private controller: AbortController | null = null;

  async *execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent> {
    this.controller = new AbortController();
    const signal = this.controller.signal;
    config.abortSignal?.addEventListener("abort", () => this.controller?.abort());

    const stream = await callMyApi(config, { signal });
    for await (const chunk of stream) {
      yield translateChunk(chunk);
    }
  }

  abort() {
    this.controller?.abort();
  }
}

Drop it onto the runtime:

const runtime = new SessionRuntime({
  provider: new MyProvider(),
  storage:  supabaseAdapter,
  userId:   "user_123",
});

Family-strict cascade with pickNextInFamily

Provider failure inside the graph cascade walks the locked family ladder rather than silently widening cross-family. The primitive is pickNextInFamily(family, currentModel, triedModels) — it returns the next rung in the same family, or null when every rung is exhausted.

// `pickNextInFamily` is host-supplied — the harness ships no model
// registry, so you author the in-family ladder walk and register it via
// the `contributeFamilyPivot` plugin hook (below). Its signature:
declare function pickNextInFamily(
  family: string,
  currentModel: string,
  triedModels: Set<string>,
): { modelId: string; callClass: string } | null;

const next = pickNextInFamily(
  "anthropic",
  "claude-opus-4-7",
  new Set(["claude-opus-4-7"]),
);
// → { modelId: "claude-sonnet-4-7", callClass: "synthesize", ... } | null

The cascade pivot in defaultAgentGraph.ts (its providerErrorCascadeAttempted and synthesisRecoveryAttempted blocks) walks pickNextInFamily in-family. When every rung exhausts, it emits [UXParity:family-exhausted], calls setFamilyExhaustedState({...}), and returns shouldContinue: false. Hosts surface a FamilyExhaustedToast (or wire contributeFamilyExhaustedSurface — see below) and the user explicitly picks another family.

Do not silently widen cross-family. That was the prod regression shape from canvas2-prompt7-audit-2026-05-18: a cross- family fallback inside the cascade doubled cost on synthesize failures and broke the family-lock invariants downstream tools depend on. null from pickNextInFamily is the explicit termination signal — surface it, don't paper over it.

Non-matrix-resolvable models (BYOK / modal-llm / unrecognized slugs) preserve legacy cross-family fallback inside the family === null branch.

Provider routing with ProviderCatalogContract

When your gateway fronts several upstream providers for one model, ProviderCatalogContract lets you tell the runtime which upstreams to prefer — and which to avoid — per call. Supply an implementation via config.providerCatalog.

interface ProviderCatalogContract {
  // Ordered upstream-provider slugs to prefer for modelId, or null for none.
  resolveProviderOrder(
    modelId: string,
    opts?: { readonly exclude?: readonly string[] },
  ): ReadonlyArray<string> | null | Promise<ReadonlyArray<string> | null>;

  // Optional warm-up so the first real call hits a warm cache.
  prefetch?(modelIds?: ReadonlyArray<string>): Promise<void>;
}

The seam attaches the returned order to the call as ProviderPreferences with allowFallbacks: true, so the gateway can still reach a non-preferred upstream when every preferred one is down. Three rules keep it from ever stalling a turn:

• Fail-soft. A throw, null, an empty array, or a rejected promise means no preference — the call takes the default path.
• Time-bounded. The lookup is bounded at ~2s; a slower one is abandoned, so a hung catalog loses its preference rather than blocking the stream.
• opts.exclude de-prioritizes upstreams for one call. The synthesize-time reroute uses it to steer a re-invoke away from an upstream that just degraded — the same recovery move as the stalled-provider pivot, but at the upstream-selection layer.

The slugs are opaque — core never inspects their shape. Your adapter maps them onto the gateway's native routing vocabulary.

Slow-cold-start planner families

Some model families run a slow planner on a cold start — slow enough that they can't win the async-planner race, so waiting the full default budget on them is pure dead time. isSlowColdStartPlannerFamily (@pleach/core/modelfamily) flags them, and the runtime holds a flagged family to a tight ~1200ms planner gate instead of the liberal default, then moves on rather than dead-waiting.

import {
  SLOW_COLD_START_PLANNER_FAMILIES,   // currently: "deepseek", "moonshot"
  isSlowColdStartPlannerFamily,
} from "@pleach/core/modelfamily";

isSlowColdStartPlannerFamily("deepseek"); // true
isSlowColdStartPlannerFamily("anthropic"); // false

Membership keys on the model family, not on your domain — the same axis as the weak-tool-schema tool-call budget. The two gates are independent: one bounds a cold planner's wait, the other curbs a weak-schema model's tool batches.

Reasoning-only completion recovery

A reasoning / chain-of-thought model (deepseek-v4, o1-class, gemini-thinking) can finish a stream cleanly having emitted reasoning but zero user-facing text and zero tool calls — or hang until the seam watchdog force-unwinds it. Treated naively both look like an outage: the turn cascades to another provider, doubling cost, and can ship an empty answer. The correct behavior is one same-model content-elicitation retry ("write your final answer directly") before declaring the model unavailable.

The seam layer owns this recovery so it fires on the graph path, gated by an opt-in DI on SessionRuntimeConfig. Unset, the seam is byte-identical to today — no recovery arm:

import type { ReasoningRecoveryStrategy } from "@pleach/core/types/strategies";

const reasoningRecovery: ReasoningRecoveryStrategy = {
  // Authoritative: is this a reasoning model eligible for recovery?
  classifier: (modelId) => modelId.startsWith("deepseek-v4"),
  // The re-ask. A STATIC string, or a FUNCTION of the empty turn's
  // own (bounded) reasoning for a CORRECTIVE directive — higher
  // recovery reliability than a cold generic re-ask.
  elicitationPrompt: (ctx) =>
    ctx.reasoningText
      ? `You reasoned:\n${ctx.reasoningText}\n\nNow write the final answer for the user directly.`
      : "Now write your final answer for the user directly. Do not include your reasoning.",
};

const runtime = new SessionRuntime({ /* … */, reasoningRecovery });

The strategy shape mirrors ProviderDegradationStatsResolver — the host owns every domain value (the model set, the elicitation string) so no reasoning-model list lives in core. The recovery fires at most once per invoke, on both the clean-empty return path and the watchdog-unwind throw path, and only cascades if the retry is also content-free. It emits [UXParity:reasoning-only-recovery:*].

The construction above is opt-in — an unset SessionRuntime gets no recovery arm. The packaged createPleachRoute surface flips that default: reasoning-only recovery is on there, so a reasoning model that finishes a stream having emitted only its chain-of-thought gets one same-model re-elicitation instead of cascading to a false family-exhausted. Pass your own strategy to override the classifier or elicitation, or recoverReasoningOnly: false to opt out.

@pleach/core/providers also publishes the shared failure taxonomy — FailureCategory (AUTH / BILLING / RATE_LIMIT / MODEL_UNAVAILABLE / TOOL_HALLUCINATION / …), FailureClassification, BillingDisposition, ModelUnavailableReason — plus classifyEmptyStream (the precondition classifier the seam recovery keys on). Cross-SKU consumers (gateway, replay, observe) share the vocabulary; the provider-specific string→category matchers stay host-injected.

Plugin contribution hooks for retry, continuation, and family pivot

Four optional HarnessPlugin contribution hooks lift the retry loop's domain knowledge out of the host. Each is opt-in; an unset hook returns null and the runtime falls back to its default behavior.

The four hooks split the responsibility cleanly: contributeRetryPolicy

• contributeContinuationPolicy carry DATA (tool name sets, regex patterns, templates); contributeFamilyPivot carries OPERATIONS (the harness has no model registry — both classification and in-family pick are host-supplied); contributeFamilyExhaustedSurface carries UI strategy (the harness has no view layer). Sketch:

import type { HarnessPlugin } from "@pleach/core";
import type { ProviderFamily } from "@pleach/core/modelfamily";

// Host-supplied — your own model-registry helpers. The harness has no
// model registry, so you author these and hand them to the runtime
// through `contributeFamilyPivot`.
declare function deriveFamilyFromModelId(modelId: string): ProviderFamily | null;
declare function pickNextInFamily(
  family: ProviderFamily,
  current: string,
  tried: Set<string>,
): { modelId: string } | null;

const myPlugin: HarnessPlugin = {
  name: "my-host",

  contributeFamilyPivot() {
    return {
      deriveFamilyFromModelId,
      pickNextInFamily: (family, current, tried) =>
        pickNextInFamily(family, current, new Set(tried))?.modelId ?? null,
    };
  },

  contributeFamilyExhaustedSurface() {
    return {
      surfaceFamilyExhausted: ({ family, triedModels, chatId, messageId }) => {
        myToastBus.emit({
          kind: "family-exhausted",
          family,
          attempted: Array.from(triedModels),
          chatId,
          messageId,
        });
      },
    };
  },
};

BYOK credential resolution is a sibling path: the createPleachRuntime factory accepts a byokResolver callback (per-session credential lookup) that the TurnOrchestrator reads via SessionRuntime.getByokResolver(). Any host with its own secrets store implements the same shape.

Provider cascade with audit attribution

Compose a primary plus fallback by wrapping two providers in a third. The cascade records why it fell through so the audit row joins back to a cause, not just a model id.

class CascadeProvider implements AgentProvider {
  constructor(private primary: AgentProvider, private fallback: AgentProvider) {}
  readonly capabilities = this.primary.capabilities;

  async *execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent> {
    try {
      yield* this.primary.execute(config);
    } catch (err) {
      yield { type: "error", error: String(err), code: "5001" };
      yield { type: "message.delta", delta: "[fallback engaged]" };
      yield* this.fallback.execute({
        ...config,
        providerConfig: { ...config.providerConfig, cascadeReason: "primary_5001" },
      });
    }
  }

  abort() { this.primary.abort(); this.fallback.abort(); }
}

Stalled provider pivot vs user abort

A provider that accepts the request but stalls — a long time-to-first-token, or a stream that opens and then starves — is a failure the cascade recovers from, not a reason to end the turn. When the watchdog unwinds a stalled rung, the runtime tears down that provider stream and pivots to the next healthy rung in the same family, exactly as it does for an outright HTTP failure. See Family-strict cascade with pickNextInFamily for the rung walk itself.

The teardown is classed SYSTEM_ABORT, not USER_ABORT. The distinction is load-bearing for audit metrics: a stalled stream cancels the same way a user pressing Stop cancels it, so without the system-abort tag every provider pivot would inflate the user-abort count. The runtime tags the teardown as a system pivot only when it, not the client, initiated it. A genuine client Stop stays USER_ABORT.

A stalled rung and a dead rung therefore land in the ledger the same shape — a providerCascade row per step (see audit attribution) — and both attribute to a cause, not just a model id. The turn only surfaces an error to the client once every in-family rung is exhausted, the family-exhausted terminal state described in Errors and recovery.

BYOK provider from session-scoped credentials

A multi-tenant deployment carries the customer key on the session, not the process env. Resolve it at construction time from the same userId / organizationId you pass to the runtime.

async function providerForOrg(organizationId: string): Promise<AgentProvider> {
  const apiKey = await loadOrgKey(organizationId);   // your secrets store
  return new AnthropicSdkProvider({
    apiKey,
    model:     "claude-sonnet-4-5",
    maxTokens: 4096,
  });
}

const runtime = new SessionRuntime({
  provider: await providerForOrg("org-acme"),
  storage:  supabaseAdapter,
  userId:   "user-7",
});

Provider vs orchestratorConfig

SessionRuntimeConfig accepts both provider (the AgentProvider-shaped surface above) and orchestratorConfig (a richer config object used by the legacy orchestrator path). Use provider for new code — it's the public substrate API. orchestratorConfig exists so hosts mid-migration don't break; new consumers should ignore it.

The class behind orchestratorConfig was renamed: OrchestratorClient → TurnOrchestrator. OrchestratorClient remains exported from @pleach/core as a deprecated alias for the @pleach/core@1.x migration window and is removed at 2.0.0. Existing imports keep working; new code should reach for TurnOrchestrator (or, better, the provider surface above).

Where to go next