Orchestrator middleware

@pleach/core/middleware wraps the orchestrator's call lifecycle with four optional interception points: before and after each model call, before and after each tool call. Middleware is where context management lives — evicting an oversized tool result to a backing store, or summarizing a long history before it blows the context window. Two middlewares ship for exactly those jobs.

import { MiddlewareStack } from "@pleach/core/middleware";
import type { OrchestratorMiddleware } from "@pleach/core/middleware";

The OrchestratorMiddleware contract

Every method is optional. Implement only the seams you need.

interface OrchestratorMiddleware {
  name: string;

  // Before each model call — reshape messages, tools, or the system prompt.
  beforeModelCall?(req: ModelRequest): Promise<ModelRequest>;

  // After each model call — rewrite or filter the response.
  afterModelCall?(resp: ModelResponse, req: ModelRequest): Promise<ModelResponse>;

  // Before each tool call — modify the call, or return null to skip it.
  beforeToolCall?(call: ToolCallRef): Promise<ToolCallRef | null>;

  // After each tool call — rewrite the result or attach context.
  afterToolCall?(call: ToolCallRef, result: ToolResultRef): Promise<ToolResultRef>;
}

ModelRequest carries the messages, tools, systemPrompt, model, temperature, maxTokens, and the resolved callClass. ModelResponse carries the content, any toolCalls, the finishReason, and usage. ToolCallRef is { id, name, args }; ToolResultRef is { toolCallId, data, isError? }.

Composing a stack

MiddlewareStack runs a list of middlewares in order. The wrap is symmetric: beforeModelCall runs first-to-last, and afterModelCall runs last-to-first, so an outer middleware sees the final response.

import { MiddlewareStack, ResultEvictionMiddleware } from "@pleach/core/middleware";

const stack = new MiddlewareStack()
  .use(new ResultEvictionMiddleware({ maxTokens: 2000, backend }));

Wire the stack through the orchestrator with setOrchestratorMiddlewareInit(() => ({ stack, hookRegistrationPromise })). With no stack wired, the runtime uses an empty one — no per-turn middleware runs.

ResultEvictionMiddleware

A tool that returns a large blob can crowd out the rest of the conversation. ResultEvictionMiddleware moves an oversized result out of the context window and leaves a compact reference in its place.

interface ResultEvictionConfig {
  maxTokens?: number;               // evict above this estimate (default 2000)
  backend: BackendWriteSurface;     // where the full result is written
  excludedTools?: readonly string[]; // tools whose results always stay in context
}

On afterToolCall, when the serialized result's estimated tokens exceed maxTokens, the middleware writes the full result through the backend and replaces the in-context payload with a reference object — { _evicted: true, preview, full_result_path, estimated_tokens, note } — carrying a head/tail preview and the path to retrieve the whole thing. Tools in excludedTools are never evicted, and a result already offloaded upstream is left alone.

SummarizationMiddleware

Where eviction handles one big result, summarization handles a long history. It compresses the older turns before the request exceeds the model's input budget.

interface SummarizationConfig {
  maxInputTokens: number;           // the model's input budget
  keepMessages?: number;            // recent turns kept verbatim (default 6)
  triggerRatio?: number;            // summarize at this fraction of budget (default 0.85)
  backend?: BackendWriteSurface;    // optional full-history offload
  threadId?: string;
  onSummarized?: (info: {
    originalTokens: number;
    summarizedTokens: number;
    messagesRemoved: number;
  }) => void;
}

On beforeModelCall, when the token count crosses maxInputTokens * triggerRatio, the middleware keeps the most recent keepMessages turns verbatim and replaces the older ones with a single extractive summary carried as a system message. If a backend is supplied, the full history is offloaded through it — best-effort, so it never blocks the turn. onSummarized fires with the before/after token counts when a pass runs.

The backend seam

Both middlewares write through the same minimal interface, so you supply whatever store fits — a file system, object storage, a database row:

interface BackendWriteSurface {
  write(path: string, content: string): Promise<unknown>;
}

Token counts are estimated with CHARS_PER_TOKEN (a value of 4), so neither middleware pulls in a tokenizer dependency to decide when to compress.

Language-model middleware

LanguageModelMiddleware is a separate, AI-SDK-v3-compatible shape for wrapping the model call at the provider layer rather than the orchestrator layer. It exposes three optional seams — transformParams (reshape the params), wrapGenerate (wrap a single generation), and wrapStream (wrap the streamed generation) — over the LanguageModelParams / GenerateResponse / StreamResponse types. Use it when you're adapting an existing AI SDK middleware; use OrchestratorMiddleware for Pleach-native before/after interception.

Where to go next

• Model hooks — the lighter pre/post-model hook seam, and where toolCallValidatorHook / repetitionDetectorHook auto-register.
• Tools — how the runtime batches the tool calls that beforeToolCall / afterToolCall wrap.
• Storage — the durable stores a BackendWriteSurface can front.