Gateway · Migration from `@pleach/core`

@pleach/core's model-family substrate is sufficient for single-tenant agents that drive their own provider keys. The gateway layers on top when the deployment shape changes — when calls need to be attributed per tenant, BYOK per tenant, cost-rolled per tenant, or routed through region / failover policies.

When to introduce the gateway

Add @pleach/gateway when any of the following is true:

• Multi-tenant. The same runtime serves multiple end customers (or multiple internal cost centers) and every call must be attributable to one of them.
• Per-tenant BYOK. Tenants supply their own provider keys for one or more families, and the platform routes via the tenant's key rather than a shared platform key.
• Cost rollup. A GROUP BY tenant_id over emitted cost events drives billing, internal showback, or quota enforcement.
• Region failover. Calls must be routed through a primary region with automatic failover to a DR region on regional outage.
• Identity federation. Provider keys are derived per-call from an OIDC / SAML / federated identity (rather than long-lived API keys).

When none of these apply, stay on @pleach/core. Don't bolt on the gateway speculatively — it adds a per-call indirection and a per-tenant client lifecycle that single-tenant agents don't need.

What stays the same

Your existing createPleachRuntime() calls do not change. The runtime's model-family resolution, in-family cascade, and provider seam contracts are unchanged. The gateway is a wrapper, not a replacement.

The matrix, the cascade walk order, the (family × callClass) cells — all of that lives in @pleach/core and the gateway re-exports CallClass, ProviderFamily, and RoutingDecision so consumers that talk only to the gateway don't need a second import line for the substrate types.

The minimum migration

A single-tenant @pleach/core consumer typically looks like this:

import { createPleachRuntime } from "@pleach/core"

const runtime = createPleachRuntime({
  providers: { /* … */ },
})

const { id: sessionId } = await runtime.sessions.create()

for await (const event of runtime.executeMessage(sessionId, "…")) {
  // handle each StreamEvent
}

The gateway-wrapped form keeps the runtime intact and introduces a per-tenant GatewayClient that resolves (tenant → credential → region → transport) ahead of the runtime call:

import { createPleachRuntime } from "@pleach/core"
import {
  GatewayClient,
  asTenantId,
} from "@pleach/gateway"
import { createCredentialRoutingMiddleware } from "@pleach/gateway"
import { createPostgresCredentialStore } from "@pleach/gateway/byok/adapters/postgres"
import { createCostMiddleware, createPostgresCostEmitter } from "@pleach/gateway/cost"

// 1. One runtime, unchanged from your single-tenant setup.
const runtime = createPleachRuntime({ providers: { /* … */ } })

// 2. Per-tenant BYOK resolution.
const credentialStore = createPostgresCredentialStore({ pool })
const resolveCredential = createCredentialRoutingMiddleware({
  store: credentialStore,
  fallbackKey: process.env.PLATFORM_ANTHROPIC_KEY,
  onMissing: "fallback",
})

// 3. Per-call cost emission.
const costEmitter = createPostgresCostEmitter({
  pool,
  table: "pleach_gateway_cost_events",
})
const recordCost = createCostMiddleware({
  emitter: costEmitter,
  computeCost: (route, response) => ({
    type: "domain.gateway.cost.recorded",
    tenantId: asTenantId(response.tenantId),
    family: response.family,
    callClass: response.callClass,
    modelInvoked: response.modelInvoked,
    costUsd: response.costUsd,
    promptTokens: response.usage.promptTokens,
    completionTokens: response.usage.completionTokens,
    byokActive: response.byokActive,
    routingDecision: response.routingDecision,
    timestamp: new Date().toISOString(),
  }),
})

// 4. One GatewayClient per tenant (SaaS pattern) or per cost center
//    (internal-enterprise pattern). transports + allowedFamilies are
//    the operator's governance hooks.
function buildGatewayForTenant(tenantId: string): GatewayClient {
  return new GatewayClient({
    tenantId: asTenantId(tenantId),
    transports: tenantTransportMap,
    allowedFamilies: tenantAllowedFamilies(tenantId),
    costEventEmitter: { emit: (e) => costEmitter.emit(e) },
  })
}

The runtime continues to do the in-family cascade math, the plan-generation rung, the streaming pipeline — everything it did before. The gateway is a layer in front of the transport seam that swaps in the tenant's credential and accounts for the call after the fact.

Per-tenant client lifecycle

One GatewayClient per tenant is the canonical pattern. The constructor captures tenantId, allowedFamilies, the transport map, and the cost-event sink — all scoped to the instance. Don't share one client across tenants; the cost attribution will collapse.

For high-tenant-count deployments, cache GatewayClient instances in an LRU keyed by tenantId. close() on eviction is a state flag today (Phase A has no underlying resources to release — transports are caller-supplied) — but call it anyway so future versions can release region-scoped pools or identity-cache entries.

Where the routing decision happens

In a single-tenant @pleach/core setup, the runtime picks the model inside its provider seam:

runtime → provider seam → @pleach/core matrix → cascade walk

With the gateway in front:

GatewayClient.route()
  → governance gate   (allowedFamilies allowlist)
  → credential gate   (BYOK middleware: tenant store → fallback)
  → transport pick    (Map<family, GatewayTransport>)
  → cascade walk      (pickNextInFamily on 503 / timeout / 429)
  → cost event        (one per successful call)

The runtime's internal cascade math is reused — gateway calls pickNextInFamily from the substrate directly. There is no duplicated walk order, no second matrix.

The bundled route factory

A bundled createGatewayRoute factory ships today at @pleach/gateway/next. It is a Web-standard (req: Request) => Promise<Response> handler (no next peer dependency — the /next subpath name is cosmetic, so it works in App Router, Workers, Bun, and Hono) that wraps a fully-built GatewayClient. It collapses the manual "resolve tenant → parse body → call route() → shape a Response" wiring into one factory:

import { createGatewayRoute } from "@pleach/gateway/next"

// Multi-tenant: a GatewayClient is constructed per request with the
// resolved tenant id, so cost events + governance are tenant-scoped.
export const POST = createGatewayRoute({
  clientOptions: {
    transports,                                  // Map<family, GatewayTransport>
    allowedFamilies: new Set(["anthropic", "openai"]),
  },
  tenantId: (req) => req.headers.get("x-tenant-id") ?? "anonymous",
})

For a single-tenant route, pass a pre-built client instead of clientOptions + tenantId:

export const POST = createGatewayRoute({ client })

The handler accepts a JSON body ({ family, callClass, model, prompt, byokKey? }) and returns the resolved GatewayResponse as one application/json object. It is NOT a streaming transport — GatewayClient.route() buffers the transport call. Supply resolveRoute(req, body) to map a different request schema onto GatewayRouteOptions. The route does NOT re-implement any routing math or the in-family cascade — those live entirely inside GatewayClient.route().

The explicit composition shown above remains available when you want every layer auditable by hand — exactly the property that matters for compliance attestation.

Cited source

• packages/gateway/src/index.ts — GatewayClient Phase A.
• packages/gateway/src/byok/CredentialRoutingMiddleware.ts — BYOK middleware factory.
• packages/gateway/src/cost/CostMiddleware.ts — cost middleware factory.

Where to go next