Providers
Wire an LLM provider into the runtime — Vercel AI SDK, Anthropic SDK, or your own — through the `AgentProvider` interface.
A provider is what the runtime invokes to talk to a model.
@pleach/core ships two — AiSdkProvider (Vercel AI SDK) and
AnthropicSdkProvider (Anthropic's official SDK) — plus one
interface (AgentProvider) you implement to wrap any other SDK.
The underlying SDK packages are optional peer deps. The runtime
imports them dynamically at provider construction, so you only
install the ones you use. See Model resolution matrix
for how (family × callClass) resolves to a concrete model id, and
Tools for how ProviderToolDef is built.
This page is the reference for the routing cluster's transport
layer. The three concepts that decide what fires through which
provider — CallClass, Seam, family-lock — are framed in
Family-lock → the routing cluster.
ProviderFamily is a closed union of seven: anthropic | openai | google | deepseek | moonshot | mistral | xai. Each family locks the
tokenizer, prompt-cache key, tool-call dialect, and refusal
pattern — see Family-lock. The transport is
one of four: native | openrouter | byok-native | byok-openrouter,
locked at session start and never silently mutated.
Same shape, different provider
The fastest swap path is AiSdkProvider + OpenRouter. The model
identifier is a <family>/<model> string — change it, change
providers. Everything else on the page stays identical.
// Anthropic
import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { AiSdkProvider, SessionRuntime } from "@pleach/core";
const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! });
const runtime = new SessionRuntime({
provider: new AiSdkProvider({
model: openrouter("anthropic/claude-sonnet-4-5"),
}),
storage: supabaseAdapter,
userId: "user_123",
});// OpenAI — same file, change one string
const runtime = new SessionRuntime({
provider: new AiSdkProvider({
model: openrouter("openai/gpt-4o"),
}),
storage: supabaseAdapter,
userId: "user_123",
});// Google Gemini — same file, change one string
const runtime = new SessionRuntime({
provider: new AiSdkProvider({
model: openrouter("google/gemini-2.5-flash"),
}),
storage: supabaseAdapter,
userId: "user_123",
});One OPENROUTER_API_KEY, seven families
(anthropic | openai | google | deepseek | moonshot | mistral | xai),
identical request shape. The runtime's family-lock reads the
<family>/ prefix and locks the tokenizer, prompt-cache key, and
tool-call dialect for the session — no silent cross-family
fallback. Want direct vendor SDKs instead? Drop the OpenRouter
wrapper for @ai-sdk/anthropic, @ai-sdk/openai,
@ai-sdk/google etc. — covered below.
import { AiSdkProvider, AnthropicSdkProvider } from "@pleach/core";
import type {
AgentProvider,
AgentExecutionConfig,
ProviderMessage,
ProviderToolDef,
ProviderCapabilities,
ProviderStreamEvent,
TokenUsage,
} from "@pleach/core/providers";AiSdkProvider (Vercel AI SDK)
Wraps streamText from ai@6.x. Right pick when you want unified
provider switching, the AI SDK's tool dialect, and the ecosystem
of community providers. The recommended default — pair with
OpenRouter to reach any family from one
key.
import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { AiSdkProvider, SessionRuntime } from "@pleach/core";
const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! });
const runtime = new SessionRuntime({
provider: new AiSdkProvider({
model: openrouter("anthropic/claude-sonnet-4-5"),
maxSteps: 5,
}),
storage: supabaseAdapter,
userId: "user_123",
});Drop the OpenRouter wrapper for a direct provider package any time
— @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, etc.:
import { anthropic } from "@ai-sdk/anthropic";
const runtime = new SessionRuntime({
provider: new AiSdkProvider({
model: anthropic("claude-sonnet-4-5"),
maxSteps: 5,
}),
storage: supabaseAdapter,
userId: "user_123",
});Config
| Field | Type | Default | Purpose |
|---|---|---|---|
model | AI SDK LanguageModel | required | Any AI SDK model factory output |
maxSteps | number | 1 | Maps to AI SDK v6's stopWhen: stepCountIs(N) |
Install the ai package plus a provider package — OpenRouter is
the default; switch to @ai-sdk/anthropic / @ai-sdk/openai /
@ai-sdk/google for direct vendor access:
npm install ai @openrouter/ai-sdk-provider # or @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, etc.AnthropicSdkProvider
Wraps @anthropic-ai/sdk directly. Right pick when you want
native Anthropic features (prompt caching, extended thinking,
tool-use beta flags) without going through a unified wrapper.
import { AnthropicSdkProvider, SessionRuntime } from "@pleach/core";
const runtime = new SessionRuntime({
provider: new AnthropicSdkProvider({
apiKey: process.env.ANTHROPIC_API_KEY!,
model: "claude-sonnet-4-5",
maxTokens: 4096,
}),
storage: supabaseAdapter,
userId: "user_123",
});Config
| Field | Type | Default | Purpose |
|---|---|---|---|
apiKey | string | required | Anthropic API key |
model | string | "claude-sonnet-4-5-20250514" | Model id |
maxTokens | number | 4096 | Output token cap per call |
baseURL | string | – | Override for proxies / regional endpoints |
npm install @anthropic-ai/sdkThe AgentProvider interface
Implement this to wrap any other SDK — OpenAI directly, a custom gateway, a local Ollama instance, anything that streams.
interface AgentProvider {
execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent>;
abort(): void;
readonly capabilities: ProviderCapabilities;
}AgentExecutionConfig
What the runtime hands you per call.
| Field | Type | Purpose |
|---|---|---|
messages | ProviderMessage[] | Normalized conversation history |
tools | ProviderToolDef[]? | Tools available this call |
systemPrompt | string? | Composed system prompt (post-composeBudgetedPrompt) |
model | string? | Specific model id when the matrix resolved one |
temperature | number? | Sampling temperature |
maxTokens | number? | Output token cap |
abortSignal | AbortSignal? | User pressed stop / turn aborted |
providerConfig | Record<string, unknown>? | Provider-specific pass-through (cache headers, beta flags) |
ProviderMessage
The normalized message shape your provider converts to its native format.
interface ProviderMessage {
id?: string;
role: "user" | "assistant" | "system" | "tool";
content: unknown;
tool_calls?: Array<{ id: string; name: string; arguments: unknown }>;
tool_call_id?: string;
name?: string;
}ProviderStreamEvent
A tagged union. The runtime adapts these into the public
StreamEvent consumers see.
type ProviderStreamEvent =
| { type: "message.start"; messageId: string; role: string }
| { type: "message.delta"; delta: string }
| { type: "message.complete"; messageId: string; usage?: TokenUsage }
| { type: "thinking.delta"; delta: string }
| { type: "thinking.complete" }
| { type: "tool.started"; toolCallId: string; toolName: string; arguments?: unknown }
| { type: "tool.completed"; toolCallId: string; toolName: string; result?: unknown }
| { type: "tool.failed"; toolCallId: string; toolName: string; error: string }
| { type: "error"; error: string; code?: string }
| { type: "done" };TokenUsage carries inputTokens, outputTokens, totalTokens,
cacheReadTokens, cacheWriteTokens — all optional, all surfaced
on message.complete.
ProviderCapabilities
Feature detection so the runtime knows what to ask for.
interface ProviderCapabilities {
streaming: boolean;
toolCalling: boolean;
thinking: boolean;
structuredOutput: boolean;
maxContextTokens: number;
parallelToolCalls: boolean;
}The runtime reads capabilities to decide what to ask for. If
toolCalling is false, tool-using sessions won't dispatch tools
through this provider; if thinking is false, thinking-delta
events are dropped.
Minimal custom provider sketch
// lib/providers/myProvider.ts
import type { AgentProvider, AgentExecutionConfig, ProviderStreamEvent } from "@pleach/core";
export class MyProvider implements AgentProvider {
readonly capabilities = {
streaming: true,
toolCalling: true,
thinking: false,
structuredOutput: false,
maxContextTokens: 128_000,
parallelToolCalls: false,
};
private controller: AbortController | null = null;
async *execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent> {
this.controller = new AbortController();
const signal = this.controller.signal;
config.abortSignal?.addEventListener("abort", () => this.controller?.abort());
const stream = await callMyApi(config, { signal });
for await (const chunk of stream) {
yield translateChunk(chunk);
}
}
abort() {
this.controller?.abort();
}
}Drop it onto the runtime:
const runtime = new SessionRuntime({
provider: new MyProvider(),
storage: supabaseAdapter,
userId: "user_123",
});Family-strict cascade with pickNextInFamily
Provider failure inside the graph cascade walks the locked family
ladder rather than silently widening cross-family. The primitive
is pickNextInFamily(family, currentModel, triedModels) — it
returns the next rung in the same family, or null when every
rung is exhausted.
// `pickNextInFamily` is host-supplied — the harness ships no model
// registry, so you author the in-family ladder walk and register it via
// the `contributeFamilyPivot` plugin hook (below). Its signature:
declare function pickNextInFamily(
family: string,
currentModel: string,
triedModels: Set<string>,
): { modelId: string; callClass: string } | null;
const next = pickNextInFamily(
"anthropic",
"claude-opus-4-7",
new Set(["claude-opus-4-7"]),
);
// → { modelId: "claude-sonnet-4-7", callClass: "synthesize", ... } | nullThe cascade pivot in defaultAgentGraph.ts (its
providerErrorCascadeAttempted and synthesisRecoveryAttempted
blocks) walks pickNextInFamily in-family. When every rung
exhausts, it emits [UXParity:family-exhausted], calls
setFamilyExhaustedState({...}), and returns shouldContinue: false. Hosts surface a FamilyExhaustedToast (or wire
contributeFamilyExhaustedSurface — see below) and the user
explicitly picks another family.
Do not silently widen cross-family. That was the prod
regression shape from canvas2-prompt7-audit-2026-05-18: a cross-
family fallback inside the cascade doubled cost on synthesize
failures and broke the family-lock invariants downstream tools
depend on. null from pickNextInFamily is the explicit
termination signal — surface it, don't paper over it.
Non-matrix-resolvable models (BYOK / modal-llm / unrecognized
slugs) preserve legacy cross-family fallback inside the
family === null branch.
Reasoning-only completion recovery
A reasoning / chain-of-thought model (deepseek-v4, o1-class, gemini-thinking) can finish a stream cleanly having emitted reasoning but zero user-facing text and zero tool calls — or hang until the seam watchdog force-unwinds it. Treated naively both look like an outage: the turn cascades to another provider, doubling cost, and can ship an empty answer. The correct behavior is one same-model content-elicitation retry ("write your final answer directly") before declaring the model unavailable.
The seam layer owns this recovery so it fires on the graph path,
gated by an opt-in DI on SessionRuntimeConfig. Unset, the seam is
byte-identical to today — no recovery arm:
import type { ReasoningRecoveryStrategy } from "@pleach/core/types/strategies";
const reasoningRecovery: ReasoningRecoveryStrategy = {
// Authoritative: is this a reasoning model eligible for recovery?
classifier: (modelId) => modelId.startsWith("deepseek-v4"),
// The re-ask. A STATIC string, or a FUNCTION of the empty turn's
// own (bounded) reasoning for a CORRECTIVE directive — higher
// recovery reliability than a cold generic re-ask.
elicitationPrompt: (ctx) =>
ctx.reasoningText
? `You reasoned:\n${ctx.reasoningText}\n\nNow write the final answer for the user directly.`
: "Now write your final answer for the user directly. Do not include your reasoning.",
};
const runtime = new SessionRuntime({ /* … */, reasoningRecovery });The strategy shape mirrors ProviderDegradationStatsResolver — the
host owns every domain value (the model set, the elicitation string)
so no reasoning-model list lives in core. The recovery fires at most
once per invoke, on both the clean-empty return path and the
watchdog-unwind throw path, and only cascades if the retry is also
content-free. It emits [UXParity:reasoning-only-recovery:*].
@pleach/core/providers also publishes the shared failure taxonomy —
FailureCategory (AUTH / BILLING / RATE_LIMIT /
MODEL_UNAVAILABLE / TOOL_HALLUCINATION / …), FailureClassification,
BillingDisposition, ModelUnavailableReason — plus classifyEmptyStream
(the precondition classifier the seam recovery keys on). Cross-SKU
consumers (gateway, replay, observe) share the vocabulary; the
provider-specific string→category matchers stay host-injected.
Plugin contribution hooks for retry, continuation, and family pivot
Four optional HarnessPlugin contribution hooks lift the retry
loop's domain knowledge out of the host. Each is opt-in; an
unset hook returns null and the runtime falls back to its
default behavior.
| Hook | Returns | Collector | What it owns |
|---|---|---|---|
contributeRetryPolicy | RetryPolicyContribution | runtime.plugins.collectRetryPolicy() | Retry behavior on transient failures — discovery-only tool set, workflow-narration nudge patterns, nudge template |
contributeContinuationPolicy | ContinuationPolicy | runtime.plugins.collectContinuationPolicy() | When a tool result needs continuation vs termination — fetch-tool name set, stuck-discovery namespace prefixes, sandbox-clause regex |
contributeFamilyPivot | FamilyPivotContribution | runtime.plugins.collectFamilyPivot() | Classify a model into a family, then pick the next in-family rung — deriveFamilyFromModelId + pickNextInFamily paired |
contributeFamilyExhaustedSurface | FamilyExhaustedSurfaceContribution | runtime.plugins.collectFamilyExhaustedSurface() | UI surface (toast, banner, modal) when the family ladder exhausts — receives family, triedModels, chatId, optional messageId |
The four hooks split the responsibility cleanly: contributeRetryPolicy
contributeContinuationPolicycarry DATA (tool name sets, regex patterns, templates);contributeFamilyPivotcarries OPERATIONS (the harness has no model registry — both classification and in-family pick are host-supplied);contributeFamilyExhaustedSurfacecarries UI strategy (the harness has no view layer). Sketch:
import type { HarnessPlugin } from "@pleach/core";
import type { ProviderFamily } from "@pleach/core/modelfamily";
// Host-supplied — your own model-registry helpers. The harness has no
// model registry, so you author these and hand them to the runtime
// through `contributeFamilyPivot`.
declare function deriveFamilyFromModelId(modelId: string): ProviderFamily | null;
declare function pickNextInFamily(
family: ProviderFamily,
current: string,
tried: Set<string>,
): { modelId: string } | null;
const myPlugin: HarnessPlugin = {
name: "my-host",
contributeFamilyPivot() {
return {
deriveFamilyFromModelId,
pickNextInFamily: (family, current, tried) =>
pickNextInFamily(family, current, new Set(tried))?.modelId ?? null,
};
},
contributeFamilyExhaustedSurface() {
return {
surfaceFamilyExhausted: ({ family, triedModels, chatId, messageId }) => {
myToastBus.emit({
kind: "family-exhausted",
family,
attempted: Array.from(triedModels),
chatId,
messageId,
});
},
};
},
};BYOK credential resolution is a sibling path: the
createPleachRuntime factory accepts a byokResolver callback
(per-session credential lookup) that the TurnOrchestrator reads
via SessionRuntime.getByokResolver(). Any host with its own
secrets store implements the same shape.
Provider cascade with audit attribution
Compose a primary plus fallback by wrapping two providers in a third. The cascade records why it fell through so the audit row joins back to a cause, not just a model id.
class CascadeProvider implements AgentProvider {
constructor(private primary: AgentProvider, private fallback: AgentProvider) {}
readonly capabilities = this.primary.capabilities;
async *execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent> {
try {
yield* this.primary.execute(config);
} catch (err) {
yield { type: "error", error: String(err), code: "5001" };
yield { type: "message.delta", delta: "[fallback engaged]" };
yield* this.fallback.execute({
...config,
providerConfig: { ...config.providerConfig, cascadeReason: "primary_5001" },
});
}
}
abort() { this.primary.abort(); this.fallback.abort(); }
}BYOK provider from session-scoped credentials
A multi-tenant deployment carries the customer key on the
session, not the process env. Resolve it at construction time
from the same userId / organizationId you pass to the runtime.
async function providerForOrg(organizationId: string): Promise<AgentProvider> {
const apiKey = await loadOrgKey(organizationId); // your secrets store
return new AnthropicSdkProvider({
apiKey,
model: "claude-sonnet-4-5",
maxTokens: 4096,
});
}
const runtime = new SessionRuntime({
provider: await providerForOrg("org-acme"),
storage: supabaseAdapter,
userId: "user-7",
});Provider vs orchestratorConfig
SessionRuntimeConfig accepts both provider (the
AgentProvider-shaped surface above) and orchestratorConfig
(a richer config object used by the legacy orchestrator path).
Use provider for new code — it's the public substrate API.
orchestratorConfig exists so hosts mid-migration don't break;
new consumers should ignore it.
The class behind orchestratorConfig was renamed: OrchestratorClient →
TurnOrchestrator. OrchestratorClient
remains exported from @pleach/core as a deprecated alias for the
@pleach/core@1.x migration window and is removed at 2.0.0.
Existing imports keep working; new code should reach for
TurnOrchestrator (or, better, the provider surface above).