diff --git a/packages/llm/ARCHITECTURE.md b/packages/llm/ARCHITECTURE.md index 0b0a828b93..4e93f0ab45 100644 --- a/packages/llm/ARCHITECTURE.md +++ b/packages/llm/ARCHITECTURE.md @@ -42,7 +42,7 @@ const program = Effect.gen(function* () { console.log(response.text) }).pipe( Effect.provide(Layer.mergeAll( - LLM.layer({ providers: [OpenAI] }), + LLM.layer(), RequestExecutor.defaultLayer, )), ) @@ -51,7 +51,7 @@ const program = Effect.gen(function* () { The public rule is: ```txt -provider helper -> model reference -> LLM.generate / LLM.stream +provider helper -> model handle -> LLM.generate / LLM.stream ``` Provider helpers should feel boring at use sites. @@ -103,7 +103,7 @@ This split is the core design choice. | Concept | Question it answers | | --- | --- | | `provider` | Who is the deployment or product surface? | -| `protocol` | Which request/response shape should the runtime use? | +| `protocol` | Which request/response shape should the runtime use? This is an open string so custom providers can add new protocol ids. | | `id` | Which model/deployment id should be sent? | | `baseURL` | Where should HTTP go? | | `apiKey`, `headers`, `queryParams`, `native` | What deployment-specific transport data is needed? | @@ -129,7 +129,7 @@ type ModelRef = { } ``` -`ModelRef` is not a provider client. It does not send requests. It is the stable, serializable description of what should be called. +`ModelRef` is the stable, serializable description of what should be called. Provider helpers also bind an in-memory adapter to the returned model handle so direct call sites do not need to manually register adapters; serialized copies fall back to `model.protocol` registry lookup. ## Terrace 3: Follow A Request @@ -140,7 +140,7 @@ At runtime, the flow is a staircase. LLM.generate({ model, prompt }) -> LLM.request(...) -> LLMClient - -> adapter selected by model.protocol + -> adapter from the model handle, or explicit registry fallback -> provider-native target payload -> HttpClientRequest -> RequestExecutor @@ -167,7 +167,7 @@ const request = LLM.request({ }) const client = LLMClient.make({ - adapters: [OpenAIResponses.adapter, OpenAIChat.adapter], + adapters: [], patches: ProviderPatch.defaults, }) @@ -177,10 +177,10 @@ const response = yield* client.generate(request)
Adapter pipeline -The adapter is selected by `request.model.protocol`. +Explicit adapters passed to `LLMClient.make(...)` win first. If no explicit adapter matches, the adapter bound to the in-memory model handle is used. If the model was serialized and revived, `LLMClient` falls back to the explicit registry keyed by `request.model.protocol`. ```ts -const adapter = adapters.get(request.model.protocol) +const adapter = adapters.get(request.model.protocol) ?? modelAdapters.get(request.model) const candidate = adapter.prepare(request) const patched = applyTargetPatches(candidate) const target = adapter.validate(patched) @@ -196,22 +196,22 @@ const events = adapter.parse(response) Keeping the current names, an `Adapter` is the runnable implementation for one registered request route. -It is selected by `model.protocol`, not by `model.provider`. +It is selected from the model handle when the provider helper created the model in the same process. Explicit adapter registration overrides that default and remains the fallback for revived models, OpenCode config bridges, and low-level tests. ```ts const adapters = new Map( - options.adapters.map((source) => [source.runtime.protocol, source.runtime] as const), + options.adapters.map((adapter) => [adapter.protocol, adapter] as const), ) -const adapter = adapters.get(request.model.protocol) +const adapter = adapters.get(request.model.protocol) ?? modelAdapters.get(request.model) ``` -That means `protocol` currently has two jobs: +That means `protocol` has two jobs only in fallback paths: | Job | Example | | --- | --- | | Describes the wire API shape | `openai-responses`, `anthropic-messages`, `gemini`. | -| Selects the runtime adapter | `LLMClient` looks up `adapters.get(request.model.protocol)`. | +| Selects the adapter after serialization | `LLMClient` looks up `adapters.get(request.model.protocol)`. | The adapter then owns the full compile/run boundary for that selected route. @@ -241,7 +241,7 @@ So the current relationship is: ```txt ModelRef.protocol - -> selects Adapter + -> selects Adapter after serialization / registry lookup -> Adapter composes Protocol + Endpoint + Auth + Framing -> Adapter compiles the request and parses the response ``` @@ -269,7 +269,7 @@ Provider behavior is split across reusable layers instead of one large provider ```txt Provider helper - creates ModelRef values + creates model handles backed by ModelRef values Provider module exports adapters and helper constructors @@ -325,8 +325,8 @@ OpenAICompatible.model("gpt-4o-mini", { provider: "local-gateway", baseURL }) | Layer | Owns | | --- | --- | -| Provider helper | Public constructor, defaults, provider identity, model capabilities, limits. | -| Provider module | Exported adapters and helpers passed to `LLM.layer({ providers })`. | +| Provider helper | Public constructor, defaults, provider identity, model capabilities, limits, in-process adapter binding. | +| Provider module | Exported adapters and helpers for explicit registry fallback. | | Adapter | Runtime registration and composition. | | Protocol | Request lowering, target schema, chunk schema, stream state machine. | | Endpoint | URL construction, base URL, path, query params, deployment routing. | @@ -394,9 +394,34 @@ The difference is below the public API. | Concern | AI SDK | This package | | --- | --- | --- | -| Use site | Provider creates runnable model object. | Provider creates `ModelRef`; `LLM` runtime runs it. | +| Use site | Provider creates runnable model object. | Provider creates a runnable model handle backed by serializable `ModelRef`. | | Provider implementation | Usually provider-package-specific language model classes. | Protocol, endpoint, auth, framing, and patches are separate axes. | | OpenAI-compatible reuse | Dedicated OpenAI-compatible implementation. | Reuses `OpenAIChat.protocol` with different deployment axes. | | Debug/replay/parity | Mostly hidden behind provider implementation. | Exposed through request lowering, patches, adapters, and events. | The tradeoff is intentional. The public API should feel small. The internals should be inspectable enough for OpenCode to preserve provider parity, replay HTTP, diff native payloads, and migrate provider-by-provider without cloning whole adapter classes. + +### OpenCode Provider Loading + +OpenCode's current AI SDK path is more dynamic than this package's native path. + +```txt +OpenCode config/models.dev + -> model.api.npm + -> import or install AI SDK provider package + -> create provider SDK + -> sdk.languageModel(...) / sdk.responses(...) / sdk.chat(...) +``` + +That is why OpenCode can point at many AI SDK provider packages without this repo shipping a native adapter for each one. + +The `@opencode-ai/llm` native path currently works in two modes: + +| Mode | How it works | Good for | +| --- | --- | --- | +| In-process model helper | `OpenAI.model(...)`, `OpenAICompatible.model(...)`, or a third-party helper returns a model handle bound to an adapter. | Library users and code that imports the provider package directly. | +| Explicit adapter registry | `LLMClient.make({ adapters: [...] })` maps revived `ModelRef.protocol` values to shipped adapters. | OpenCode config/models.dev bridges, tests, request replay, serialized models. | + +So OpenCode native integration is not “import any AI SDK provider package and it just works” yet. Today it supports the protocols/providers we can resolve to known native adapters, plus generic OpenAI-compatible deployments. A config-defined provider with `@ai-sdk/openai-compatible` can resolve to `openai-compatible-chat`; a brand-new protocol needs a native adapter and resolver mapping. + +The core package is now open enough for external protocols: `ProtocolID` is just a string, so a third-party package can define `Protocol.define(...)`, `Adapter.fromProtocol(...)`, and a model helper without changing this package. To make OpenCode load those from config the same way it loads AI SDK packages, we would add an explicit native-provider loader/registry analogous to the AI SDK `model.api.npm` loader. diff --git a/packages/llm/example/tutorial.ts b/packages/llm/example/tutorial.ts index ed737335f3..11edc20f69 100644 --- a/packages/llm/example/tutorial.ts +++ b/packages/llm/example/tutorial.ts @@ -96,10 +96,9 @@ const FakeTarget = Schema.Struct({ type FakeTarget = Schema.Schema.Type const FakeProtocol = Protocol.define({ - // ProtocolID is a closed union in this package. A real new provider protocol - // would add its own id there; this tutorial reuses `openai-chat` so the fake - // provider can compile without changing production protocol ids. - id: "openai-chat", + // Protocol ids are open strings, so external packages can define their own + // protocols without changing this package. + id: "fake-echo", target: FakeTarget, prepare: (request) => Effect.succeed({ @@ -129,16 +128,19 @@ const FakeAdapter = Adapter.fromProtocol({ framing: Framing.sse, }) -// A provider module exports adapters plus model helpers. The model helper sets -// provider identity and the protocol id used for adapter lookup. +// A provider module exports a model helper. The model helper sets provider +// identity, protocol id, and the adapter that can run this in-memory model +// handle. Serialized / revived models can still use explicit provider adapters. const FakeEcho = { - adapters: [FakeAdapter], model: (id: string) => - LLM.model({ - id, - provider: "fake-echo", - protocol: "openai-chat", - }), + Adapter.bindModel( + LLM.model({ + id, + provider: "fake-echo", + protocol: "fake-echo", + }), + FakeAdapter, + ), } // `prepare` compiles through patches, protocol lowering, validation, endpoint, @@ -152,7 +154,7 @@ const inspectFakeProvider = Effect.gen(function* () { console.log("\n== fake provider prepare ==") console.log("adapter:", prepared.adapter) console.log("target:", Formatter.formatJson(prepared.target, { space: 2 })) -}).pipe(Effect.provide(LLM.layer({ providers: [FakeEcho] }))) +}).pipe(Effect.provide(LLM.layer())) // Provide the LLM runtime and the HTTP request executor once. The default path // sends one live generate call and one local fake-provider prepare call. @@ -163,6 +165,6 @@ const program = Effect.gen(function* () { yield* inspectFakeProvider // yield* streamText // yield* streamWithTools -}).pipe(Effect.provide(Layer.mergeAll(LLM.layer({ providers: [OpenAI] }), RequestExecutor.defaultLayer))) +}).pipe(Effect.provide(Layer.mergeAll(LLM.layer(), RequestExecutor.defaultLayer))) Effect.runPromise(program) diff --git a/packages/llm/src/adapter.ts b/packages/llm/src/adapter.ts index bab2453dfd..f6c7ecebea 100644 --- a/packages/llm/src/adapter.ts +++ b/packages/llm/src/adapter.ts @@ -21,49 +21,50 @@ import type { } from "./schema" import { LLMRequest as LLMRequestSchema, LLMResponse, NoAdapterError, PreparedRequest as PreparedRequestSchema } from "./schema" -interface RuntimeAdapter { - readonly id: string - readonly protocol: ProtocolID - readonly patches: ReadonlyArray> - readonly prepare: (request: LLMRequest) => Effect.Effect - readonly validate: (draft: unknown) => Effect.Effect - readonly toHttp: (target: unknown, context: HttpContext) => Effect.Effect - readonly parse: (response: HttpClientResponse.HttpClientResponse, context: HttpContext) => Stream.Stream -} - -interface RuntimeAdapterSource { - readonly runtime: RuntimeAdapter -} - export interface HttpContext { readonly request: LLMRequest readonly patchTrace: ReadonlyArray } -export interface Adapter { +export interface Adapter { readonly id: string readonly protocol: ProtocolID - readonly patches: ReadonlyArray> - readonly prepare: (request: LLMRequest) => Effect.Effect - readonly validate: (draft: Draft) => Effect.Effect + readonly patches: ReadonlyArray> + readonly prepare: (request: LLMRequest) => Effect.Effect + readonly validate: (target: Target) => Effect.Effect readonly toHttp: (target: Target, context: HttpContext) => Effect.Effect readonly parse: (response: HttpClientResponse.HttpClientResponse, context: HttpContext) => Stream.Stream } -export interface AdapterInput { - readonly id: string - readonly protocol: ProtocolID - readonly patches?: ReadonlyArray> - readonly prepare: (request: LLMRequest) => Effect.Effect - readonly validate: (draft: Draft) => Effect.Effect - readonly toHttp: (target: Target, context: HttpContext) => Effect.Effect - readonly parse: (response: HttpClientResponse.HttpClientResponse, context: HttpContext) => Stream.Stream +export type AdapterInput = Omit, "patches"> & { + readonly patches?: ReadonlyArray> } -export interface AdapterDefinition extends Adapter { - readonly runtime: RuntimeAdapter - readonly patch: (id: string, input: PatchInput) => Patch - readonly withPatches: (patches: ReadonlyArray>) => AdapterDefinition +export interface AdapterDefinition extends Adapter { + readonly patch: (id: string, input: PatchInput) => Patch + readonly withPatches: (patches: ReadonlyArray>) => AdapterDefinition +} + +// Adapter registries intentionally erase target generics after the typed +// adapter is constructed. This keeps normal call sites on `OpenAIChat.adapter` +// instead of leaking a separate runtime-adapter wrapper. +// oxlint-disable-next-line typescript-eslint/no-explicit-any +export type AnyAdapter = AdapterDefinition + +const modelAdapters = new WeakMap() + +export const bindModel = (model: Model, adapter: AnyAdapter): Model => { + if (model.protocol !== adapter.protocol) { + throw new Error(`Cannot bind ${adapter.id} adapter (${adapter.protocol}) to ${model.provider}/${model.id} (${model.protocol})`) + } + modelAdapters.set(model, adapter) + return model +} + +export const preserveModelBinding = (source: ModelRef, target: Model): Model => { + const adapter = modelAdapters.get(source) + if (!adapter) return target + return bindModel(target, adapter) } export interface LLMClient { @@ -85,7 +86,7 @@ export interface LLMClient { } export interface ClientOptions { - readonly adapters: ReadonlyArray + readonly adapters?: ReadonlyArray readonly patches?: PatchRegistry | ReadonlyArray } @@ -108,16 +109,11 @@ const normalizeRegistry = (patches: PatchRegistry | ReadonlyArray | un * canonical path is `Adapter.fromProtocol(...)`. New adapters should start * there and prove they need otherwise before reaching for this. */ -export function unsafe(input: AdapterInput): AdapterDefinition { - const build = (patches: ReadonlyArray>): AdapterDefinition => ({ +export function unsafe(input: AdapterInput): AdapterDefinition { + const build = (patches: ReadonlyArray>): AdapterDefinition => ({ id: input.id, protocol: input.protocol, patches, - get runtime() { - // Runtime registry erases adapter draft/target generics after validation. - // oxlint-disable-next-line typescript-eslint/no-unsafe-type-assertion - return this as unknown as RuntimeAdapter - }, prepare: input.prepare, validate: input.validate, toHttp: input.toHttp, @@ -175,7 +171,7 @@ export interface FromProtocolInput { */ export function fromProtocol( input: FromProtocolInput, -): AdapterDefinition { +): AdapterDefinition { const auth = input.auth ?? authBearer const protocol = input.protocol const validateTarget = ProviderShared.validateWith(Schema.decodeUnknownEffect(protocol.target)) @@ -233,12 +229,10 @@ export function fromProtocol( const makeClient = (options: ClientOptions): LLMClient => { const registry = normalizeRegistry(options.patches) - const adapters = new Map( - options.adapters.map((source) => [source.runtime.protocol, source.runtime] as const), - ) + const adapters = new Map((options.adapters ?? []).map((adapter) => [adapter.protocol, adapter] as const)) const compile = Effect.fn("LLM.compile")(function* (request: LLMRequest) { - const adapter = adapters.get(request.model.protocol) + const adapter = adapters.get(request.model.protocol) ?? modelAdapters.get(request.model) if (!adapter) return yield* noAdapter(request.model) const requestPlan = plan({ @@ -266,13 +260,13 @@ const makeClient = (options: ClientOptions): LLMClient => { tools: requestBeforeToolPatches.tools.map(toolSchemaPlan.apply), }) const patchContext = context({ request: patchedRequest }) - const draft = yield* adapter.prepare(patchedRequest) + const candidate = yield* adapter.prepare(patchedRequest) const targetPlan = plan({ phase: "target", context: patchContext, patches: [...adapter.patches, ...registry.target], }) - const target = yield* adapter.validate(targetPlan.apply(draft)) + const target = yield* adapter.validate(targetPlan.apply(candidate)) const targetPatchTrace = [ ...requestPlan.trace, ...promptPlan.trace, diff --git a/packages/llm/src/index.ts b/packages/llm/src/index.ts index ba119f0fb2..2815d1a5e5 100644 --- a/packages/llm/src/index.ts +++ b/packages/llm/src/index.ts @@ -35,6 +35,7 @@ export { GitHubCopilot } from "./provider/github-copilot" export { OpenAIChat } from "./provider/openai-chat" export { OpenAICompatibleChat } from "./provider/openai-compatible-chat" export { OpenAICompatibleFamily } from "./provider/openai-compatible-family" +export { OpenAICompatibleProfiles } from "./provider/openai-compatible-profile" export { OpenAIResponses } from "./provider/openai-responses" export { ProviderResolver } from "./provider-resolver" export { OpenAI } from "./provider/openai" diff --git a/packages/llm/src/llm.ts b/packages/llm/src/llm.ts index 7c7a9a905e..18cde1a2d2 100644 --- a/packages/llm/src/llm.ts +++ b/packages/llm/src/llm.ts @@ -1,5 +1,5 @@ import { Context, Effect, Layer, Stream } from "effect" -import { LLMClient, type ClientOptions } from "./adapter" +import { LLMClient, preserveModelBinding, type AnyAdapter, type ClientOptions } from "./adapter" import type { RequestExecutor } from "./executor" import { ProviderPatch } from "./provider/patch" import { type Tools } from "./tool" @@ -29,7 +29,7 @@ import { import type { LLMError, PreparedRequestOf } from "./schema" export interface Provider { - readonly adapters: ClientOptions["adapters"] + readonly adapters: ReadonlyArray } export interface MakeOptions { @@ -50,15 +50,13 @@ export interface Runtime { export class Service extends Context.Service()("@opencode/LLM") {} const clientOptions = (options: MakeOptions): ClientOptions => ({ - adapters: [...(options.adapters ?? []), ...(options.providers ?? []).flatMap((provider) => provider.adapters)].filter( - (source, index, all) => all.findIndex((item) => item.runtime.protocol === source.runtime.protocol) === index, - ), + adapters: [...(options.providers ?? []).flatMap((provider) => provider.adapters), ...(options.adapters ?? [])], patches: options.patches ?? ProviderPatch.defaults, }) const requestOf = (input: LLMRequest | RequestInput) => input instanceof LLMRequest ? input : request(input) -export const make = (options: MakeOptions): Runtime => { +export const make = (options: MakeOptions = {}): Runtime => { const client = LLMClient.make(clientOptions(options)) return { prepare: (input) => client.prepare(requestOf(input)), @@ -71,7 +69,7 @@ export const make = (options: MakeOptions): Runtime => { } } -export const layer = (options: MakeOptions): Layer.Layer => +export const layer = (options: MakeOptions = {}): Layer.Layer => Layer.succeed(Service, Service.of(make(options))) export const prepare = (input: LLMRequest | RequestInput) => @@ -253,7 +251,7 @@ export const requestInput = (input: LLMRequest): RequestInput => ({ export const request = (input: RequestInput) => { const { system: requestSystem, prompt, messages, tools, toolChoice: requestToolChoice, generation: requestGeneration, ...rest } = input - return new LLMRequest({ + const result = new LLMRequest({ ...rest, system: systemParts(requestSystem), messages: [...(messages?.map(message) ?? []), ...(prompt === undefined ? [] : [user(prompt)])], @@ -261,6 +259,8 @@ export const request = (input: RequestInput) => { toolChoice: requestToolChoice ? toolChoice(requestToolChoice) : undefined, generation: generation(requestGeneration), }) + preserveModelBinding(input.model, result.model) + return result } export const updateRequest = (input: LLMRequest, patch: Partial) => diff --git a/packages/llm/src/provider/anthropic-messages.ts b/packages/llm/src/provider/anthropic-messages.ts index f1f46ee6b3..be2c71c89d 100644 --- a/packages/llm/src/provider/anthropic-messages.ts +++ b/packages/llm/src/provider/anthropic-messages.ts @@ -513,16 +513,19 @@ export const adapter = Adapter.fromProtocol({ }) export const model = (input: AnthropicMessagesModelInput) => - llmModel({ - ...input, - provider: "anthropic", - protocol: "anthropic-messages", - capabilities: input.capabilities ?? capabilities({ - output: { reasoning: true }, - tools: { calls: true, streamingInput: true }, - cache: { prompt: true, contentBlocks: true }, - reasoning: { efforts: ["low", "medium", "high", "xhigh", "max"], summaries: false, encryptedContent: true }, + Adapter.bindModel( + llmModel({ + ...input, + provider: "anthropic", + protocol: "anthropic-messages", + capabilities: input.capabilities ?? capabilities({ + output: { reasoning: true }, + tools: { calls: true, streamingInput: true }, + cache: { prompt: true, contentBlocks: true }, + reasoning: { efforts: ["low", "medium", "high", "xhigh", "max"], summaries: false, encryptedContent: true }, + }), }), - }) + adapter, + ) export * as AnthropicMessages from "./anthropic-messages" diff --git a/packages/llm/src/provider/bedrock-converse.ts b/packages/llm/src/provider/bedrock-converse.ts index 937457eb30..a4baa7d366 100644 --- a/packages/llm/src/provider/bedrock-converse.ts +++ b/packages/llm/src/provider/bedrock-converse.ts @@ -1,11 +1,8 @@ -import { EventStreamCodec } from "@smithy/eventstream-codec" -import { fromUtf8, toUtf8 } from "@smithy/util-utf8" import { AwsV4Signer } from "aws4fetch" -import { Effect, Option, Schema, Stream } from "effect" +import { Effect, Option, Schema } from "effect" import { Adapter } from "../adapter" import { Auth } from "../auth" import { Endpoint } from "../endpoint" -import type { Framing } from "../framing" import { capabilities, model as llmModel, type ModelInput } from "../llm" import { Protocol } from "../protocol" import { @@ -15,11 +12,11 @@ import { type LLMEvent, type LLMRequest, type MediaPart, - type ProviderChunkError, type ToolCallPart, type ToolDefinition, type ToolResultPart, } from "../schema" +import { BedrockEventStream } from "./bedrock-event-stream" import { ProviderShared } from "./shared" const ADAPTER = "bedrock-converse" @@ -679,87 +676,7 @@ const processChunk = (state: ParserState, chunk: BedrockChunk) => return [state, []] as const }) -// Bedrock streams responses using the AWS event stream binary protocol — each -// frame is `[length:4][headers-length:4][prelude-crc:4][headers][payload][crc:4]`. -// We use `@smithy/eventstream-codec` to validate framing and CRCs, then -// reconstruct the JSON wrapping by `:event-type` so the chunk schema can match. -const eventCodec = new EventStreamCodec(toUtf8, fromUtf8) -const utf8 = new TextDecoder() - -// Cursor-tracking buffer state. Bytes accumulate in `buffer`; `offset` is the -// read position. Reading by `subarray` is zero-copy. We only allocate a fresh -// buffer when (a) a new network chunk arrives and we need to append, or (b) -// the consumed prefix is more than half the buffer (compaction). -interface FrameBufferState { - readonly buffer: Uint8Array - readonly offset: number -} - -const initialFrameBuffer: FrameBufferState = { buffer: new Uint8Array(0), offset: 0 } - -const appendChunk = (state: FrameBufferState, chunk: Uint8Array): FrameBufferState => { - const remaining = state.buffer.length - state.offset - // Compact: drop the consumed prefix and append the new chunk in one alloc. - // This bounds buffer growth to at most one network chunk past the live - // window, regardless of stream length. - const next = new Uint8Array(remaining + chunk.length) - next.set(state.buffer.subarray(state.offset), 0) - next.set(chunk, remaining) - return { buffer: next, offset: 0 } -} - -const consumeFrames = (state: FrameBufferState, chunk: Uint8Array) => - Effect.gen(function* () { - let cursor = appendChunk(state, chunk) - const out: object[] = [] - while (cursor.buffer.length - cursor.offset >= 4) { - const view = cursor.buffer.subarray(cursor.offset) - const totalLength = new DataView(view.buffer, view.byteOffset, view.byteLength).getUint32(0, false) - if (view.length < totalLength) break - - const decoded = yield* Effect.try({ - try: () => eventCodec.decode(view.subarray(0, totalLength)), - catch: (error) => - ProviderShared.chunkError( - ADAPTER, - `Failed to decode Bedrock Converse event-stream frame: ${ - error instanceof Error ? error.message : String(error) - }`, - ), - }) - cursor = { buffer: cursor.buffer, offset: cursor.offset + totalLength } - - if (decoded.headers[":message-type"]?.value !== "event") continue - const eventType = decoded.headers[":event-type"]?.value - if (typeof eventType !== "string") continue - const payload = utf8.decode(decoded.body) - if (!payload) continue - // The AWS event stream pads short payloads with a `p` field. Drop it - // before handing the object to the chunk schema. JSON decode goes - // through the shared Schema-driven codec to satisfy the package rule - // against ad-hoc `JSON.parse` calls. - const parsed = (yield* ProviderShared.parseJson( - ADAPTER, - payload, - "Failed to parse Bedrock Converse event-stream payload", - )) as Record - delete parsed.p - out.push({ [eventType]: parsed }) - } - return [cursor, out] as const - }) - -/** - * AWS event-stream framing for Bedrock Converse. Each frame is decoded by - * `@smithy/eventstream-codec` (length + header + payload + CRC) and rewrapped - * under its `:event-type` header so the chunk schema can match the JSON - * payload directly. Reusable for any AWS service that wraps JSON payloads in - * event-stream frames keyed by `:event-type`. - */ -const framing: Framing = { - id: "aws-event-stream", - frame: (bytes) => bytes.pipe(Stream.mapAccumEffect(() => initialFrameBuffer, consumeFrames)), -} +const framing = BedrockEventStream.framing(ADAPTER) // If a stream ends after `messageStop` but before `metadata` (rare but // possible on truncated transports), still surface a terminal finish. @@ -803,25 +720,28 @@ export const adapter = Adapter.fromProtocol({ export const model = (input: BedrockConverseModelInput) => { const { credentials, ...rest } = input - return llmModel({ - ...rest, - provider: "bedrock", - protocol: "bedrock-converse", - capabilities: - input.capabilities ?? - capabilities({ - output: { reasoning: true }, - tools: { calls: true, streamingInput: true }, - cache: { prompt: true, contentBlocks: true }, - }), - native: credentials - ? { - ...input.native, - aws_credentials: credentials, - aws_region: credentials.region, - } - : input.native, - }) + return Adapter.bindModel( + llmModel({ + ...rest, + provider: "bedrock", + protocol: "bedrock-converse", + capabilities: + input.capabilities ?? + capabilities({ + output: { reasoning: true }, + tools: { calls: true, streamingInput: true }, + cache: { prompt: true, contentBlocks: true }, + }), + native: credentials + ? { + ...input.native, + aws_credentials: credentials, + aws_region: credentials.region, + } + : input.native, + }), + adapter, + ) } export * as BedrockConverse from "./bedrock-converse" diff --git a/packages/llm/src/provider/bedrock-event-stream.ts b/packages/llm/src/provider/bedrock-event-stream.ts new file mode 100644 index 0000000000..ef28b72d5d --- /dev/null +++ b/packages/llm/src/provider/bedrock-event-stream.ts @@ -0,0 +1,87 @@ +import { EventStreamCodec } from "@smithy/eventstream-codec" +import { fromUtf8, toUtf8 } from "@smithy/util-utf8" +import { Effect, Stream } from "effect" +import type { Framing } from "../framing" +import { ProviderShared } from "./shared" + +// Bedrock streams responses using the AWS event stream binary protocol — each +// frame is `[length:4][headers-length:4][prelude-crc:4][headers][payload][crc:4]`. +// We use `@smithy/eventstream-codec` to validate framing and CRCs, then +// reconstruct the JSON wrapping by `:event-type` so the chunk schema can match. +const eventCodec = new EventStreamCodec(toUtf8, fromUtf8) +const utf8 = new TextDecoder() + +// Cursor-tracking buffer state. Bytes accumulate in `buffer`; `offset` is the +// read position. Reading by `subarray` is zero-copy. We only allocate a fresh +// buffer when a new network chunk arrives and we need to append. +interface FrameBufferState { + readonly buffer: Uint8Array + readonly offset: number +} + +const initialFrameBuffer: FrameBufferState = { buffer: new Uint8Array(0), offset: 0 } + +const appendChunk = (state: FrameBufferState, chunk: Uint8Array): FrameBufferState => { + const remaining = state.buffer.length - state.offset + // Compact: drop the consumed prefix and append the new chunk in one alloc. + // This bounds buffer growth to at most one network chunk past the live + // window, regardless of stream length. + const next = new Uint8Array(remaining + chunk.length) + next.set(state.buffer.subarray(state.offset), 0) + next.set(chunk, remaining) + return { buffer: next, offset: 0 } +} + +const consumeFrames = (adapter: string) => (state: FrameBufferState, chunk: Uint8Array) => + Effect.gen(function* () { + let cursor = appendChunk(state, chunk) + const out: object[] = [] + while (cursor.buffer.length - cursor.offset >= 4) { + const view = cursor.buffer.subarray(cursor.offset) + const totalLength = new DataView(view.buffer, view.byteOffset, view.byteLength).getUint32(0, false) + if (view.length < totalLength) break + + const decoded = yield* Effect.try({ + try: () => eventCodec.decode(view.subarray(0, totalLength)), + catch: (error) => + ProviderShared.chunkError( + adapter, + `Failed to decode Bedrock Converse event-stream frame: ${ + error instanceof Error ? error.message : String(error) + }`, + ), + }) + cursor = { buffer: cursor.buffer, offset: cursor.offset + totalLength } + + if (decoded.headers[":message-type"]?.value !== "event") continue + const eventType = decoded.headers[":event-type"]?.value + if (typeof eventType !== "string") continue + const payload = utf8.decode(decoded.body) + if (!payload) continue + // The AWS event stream pads short payloads with a `p` field. Drop it + // before handing the object to the chunk schema. JSON decode goes + // through the shared Schema-driven codec to satisfy the package rule + // against ad-hoc `JSON.parse` calls. + const parsed = (yield* ProviderShared.parseJson( + adapter, + payload, + "Failed to parse Bedrock Converse event-stream payload", + )) as Record + delete parsed.p + out.push({ [eventType]: parsed }) + } + return [cursor, out] as const + }) + +/** + * AWS event-stream framing for Bedrock Converse. Each frame is decoded by + * `@smithy/eventstream-codec` (length + header + payload + CRC) and rewrapped + * under its `:event-type` header so the chunk schema can match the JSON + * payload directly. + */ +export const framing = (adapter: string): Framing => ({ + id: "aws-event-stream", + frame: (bytes) => bytes.pipe(Stream.mapAccumEffect(() => initialFrameBuffer, consumeFrames(adapter))), +}) + +export * as BedrockEventStream from "./bedrock-event-stream" diff --git a/packages/llm/src/provider/gemini.ts b/packages/llm/src/provider/gemini.ts index b12346fbca..fac4018fc9 100644 --- a/packages/llm/src/provider/gemini.ts +++ b/packages/llm/src/provider/gemini.ts @@ -469,16 +469,19 @@ export const adapter = Adapter.fromProtocol({ }) export const model = (input: GeminiModelInput) => - llmModel({ - ...input, - provider: "google", - protocol: "gemini", - capabilities: input.capabilities ?? capabilities({ - input: { image: true, audio: true, video: true, pdf: true }, - output: { reasoning: true }, - tools: { calls: true }, - reasoning: { efforts: ["minimal", "low", "medium", "high", "xhigh", "max"] }, + Adapter.bindModel( + llmModel({ + ...input, + provider: "google", + protocol: "gemini", + capabilities: input.capabilities ?? capabilities({ + input: { image: true, audio: true, video: true, pdf: true }, + output: { reasoning: true }, + tools: { calls: true }, + reasoning: { efforts: ["minimal", "low", "medium", "high", "xhigh", "max"] }, + }), }), - }) + adapter, + ) export * as Gemini from "./gemini" diff --git a/packages/llm/src/provider/openai-chat.ts b/packages/llm/src/provider/openai-chat.ts index df3aeeafbb..7fcaa9a6b1 100644 --- a/packages/llm/src/provider/openai-chat.ts +++ b/packages/llm/src/provider/openai-chat.ts @@ -347,12 +347,15 @@ export const adapter = Adapter.fromProtocol({ }) export const model = (input: OpenAIChatModelInput) => - llmModel({ - ...input, - provider: "openai", - protocol: "openai-chat", - capabilities: input.capabilities ?? capabilities({ tools: { calls: true, streamingInput: true } }), - }) + Adapter.bindModel( + llmModel({ + ...input, + provider: "openai", + protocol: "openai-chat", + capabilities: input.capabilities ?? capabilities({ tools: { calls: true, streamingInput: true } }), + }), + adapter, + ) export const includeUsage = adapter.patch("include-usage", { reason: "request final usage chunk from OpenAI Chat streaming responses", diff --git a/packages/llm/src/provider/openai-compatible-chat.ts b/packages/llm/src/provider/openai-compatible-chat.ts index e4a6362ac1..fffe05dd23 100644 --- a/packages/llm/src/provider/openai-compatible-chat.ts +++ b/packages/llm/src/provider/openai-compatible-chat.ts @@ -3,7 +3,7 @@ import { Endpoint } from "../endpoint" import { Framing } from "../framing" import { capabilities, model as llmModel, type ModelInput } from "../llm" import { OpenAIChat } from "./openai-chat" -import { families, type ProviderFamily } from "./openai-compatible-family" +import { families, type OpenAICompatibleProfile } from "./openai-compatible-profile" const ADAPTER = "openai-compatible-chat" @@ -38,30 +38,39 @@ export const adapter = Adapter.fromProtocol({ }) export const model = (input: OpenAICompatibleChatModelInput) => - llmModel({ - ...input, - protocol: "openai-compatible-chat", - capabilities: input.capabilities ?? capabilities({ tools: { calls: true, streamingInput: true } }), - }) + Adapter.bindModel( + llmModel({ + ...input, + protocol: "openai-compatible-chat", + capabilities: input.capabilities ?? capabilities({ tools: { calls: true, streamingInput: true } }), + }), + adapter, + ) -const familyModel = (family: ProviderFamily, input: ProviderFamilyModelInput) => +const profileBaseURL = (profile: OpenAICompatibleProfile, input: ProviderFamilyModelInput) => { + const baseURL = input.baseURL ?? profile.baseURL + if (baseURL) return baseURL + throw new Error(`OpenAI-compatible profile ${profile.provider} requires a baseURL`) +} + +export const profileModel = (profile: OpenAICompatibleProfile, input: ProviderFamilyModelInput) => model({ ...input, - provider: family.provider, - baseURL: input.baseURL ?? family.baseURL, + provider: profile.provider, + baseURL: profileBaseURL(profile, input), }) -export const baseten = (input: ProviderFamilyModelInput) => familyModel(families.baseten, input) +export const baseten = (input: ProviderFamilyModelInput) => profileModel(families.baseten, input) -export const cerebras = (input: ProviderFamilyModelInput) => familyModel(families.cerebras, input) +export const cerebras = (input: ProviderFamilyModelInput) => profileModel(families.cerebras, input) -export const deepinfra = (input: ProviderFamilyModelInput) => familyModel(families.deepinfra, input) +export const deepinfra = (input: ProviderFamilyModelInput) => profileModel(families.deepinfra, input) -export const deepseek = (input: ProviderFamilyModelInput) => familyModel(families.deepseek, input) +export const deepseek = (input: ProviderFamilyModelInput) => profileModel(families.deepseek, input) -export const fireworks = (input: ProviderFamilyModelInput) => familyModel(families.fireworks, input) +export const fireworks = (input: ProviderFamilyModelInput) => profileModel(families.fireworks, input) -export const togetherai = (input: ProviderFamilyModelInput) => familyModel(families.togetherai, input) +export const togetherai = (input: ProviderFamilyModelInput) => profileModel(families.togetherai, input) export const includeUsage = adapter.patch("include-usage", { reason: "request final usage chunk from OpenAI-compatible Chat streaming responses", diff --git a/packages/llm/src/provider/openai-compatible-family.ts b/packages/llm/src/provider/openai-compatible-family.ts index 19435cd7fe..16922505a5 100644 --- a/packages/llm/src/provider/openai-compatible-family.ts +++ b/packages/llm/src/provider/openai-compatible-family.ts @@ -1,36 +1,10 @@ -import { ProviderResolver } from "../provider-resolver" +import { families, familyByProvider, familyResolver, resolveFamily } from "./openai-compatible-profile" +import type { OpenAICompatibleProfile } from "./openai-compatible-profile" -export interface ProviderFamily { - readonly provider: string - readonly baseURL: string -} - -export const families = { - baseten: { provider: "baseten", baseURL: "https://inference.baseten.co/v1" }, - cerebras: { provider: "cerebras", baseURL: "https://api.cerebras.ai/v1" }, - deepinfra: { provider: "deepinfra", baseURL: "https://api.deepinfra.com/v1/openai" }, - deepseek: { provider: "deepseek", baseURL: "https://api.deepseek.com/v1" }, - fireworks: { provider: "fireworks", baseURL: "https://api.fireworks.ai/inference/v1" }, - togetherai: { provider: "togetherai", baseURL: "https://api.together.xyz/v1" }, -} as const satisfies Record - -export const byProvider: Record = Object.fromEntries( - Object.values(families).map((family) => [family.provider, family]), -) - -const resolutions = Object.fromEntries( - Object.values(families).map((family) => [ - family.provider, - ProviderResolver.make(family.provider, "openai-compatible-chat", { baseURL: family.baseURL }), - ]), -) - -export const resolve = (provider: string) => - resolutions[provider] ?? ProviderResolver.make(provider, "openai-compatible-chat") - -export const resolver = ProviderResolver.define({ - id: ProviderResolver.make("openai-compatible", "openai-compatible-chat").provider, - resolve: (input) => resolve(input.providerID), -}) +export type ProviderFamily = OpenAICompatibleProfile +export const byProvider = familyByProvider +export const resolve = resolveFamily +export const resolver = familyResolver +export { families } export * as OpenAICompatibleFamily from "./openai-compatible-family" diff --git a/packages/llm/src/provider/openai-compatible-profile.ts b/packages/llm/src/provider/openai-compatible-profile.ts new file mode 100644 index 0000000000..8f94acd9da --- /dev/null +++ b/packages/llm/src/provider/openai-compatible-profile.ts @@ -0,0 +1,68 @@ +import type { CapabilitiesInput } from "../llm" +import { ProviderResolver, type ProviderResolution } from "../provider-resolver" + +export interface OpenAICompatibleProfile { + readonly provider: string + readonly baseURL?: string + readonly capabilities?: CapabilitiesInput + readonly resolver?: Partial> +} + +export const families = { + baseten: { provider: "baseten", baseURL: "https://inference.baseten.co/v1" }, + cerebras: { provider: "cerebras", baseURL: "https://api.cerebras.ai/v1" }, + deepinfra: { provider: "deepinfra", baseURL: "https://api.deepinfra.com/v1/openai" }, + deepseek: { provider: "deepseek", baseURL: "https://api.deepseek.com/v1" }, + fireworks: { provider: "fireworks", baseURL: "https://api.fireworks.ai/inference/v1" }, + togetherai: { provider: "togetherai", baseURL: "https://api.together.xyz/v1" }, +} as const satisfies Record + +export const profiles = { + ...families, + openrouter: { provider: "openrouter", baseURL: "https://openrouter.ai/api/v1" }, +} as const satisfies Record + +export const familyByProvider: Record = Object.fromEntries( + Object.values(families).map((profile) => [profile.provider, profile]), +) + +export const byProvider: Record = Object.fromEntries( + Object.values(profiles).map((profile) => [profile.provider, profile]), +) + +export const resolution = (profile: OpenAICompatibleProfile) => + ProviderResolver.make(profile.provider, "openai-compatible-chat", { + baseURL: profile.baseURL, + capabilities: profile.capabilities, + ...profile.resolver, + }) + +export const resolve = (provider: string) => { + const profile = byProvider[provider] + if (profile) return resolution(profile) + return ProviderResolver.make(provider, "openai-compatible-chat") +} + +export const resolveFamily = (provider: string) => { + const profile = familyByProvider[provider] + if (profile) return resolution(profile) + return ProviderResolver.make(provider, "openai-compatible-chat") +} + +export const resolverFor = (profile: OpenAICompatibleProfile) => + ProviderResolver.define({ + id: ProviderResolver.make(profile.provider, "openai-compatible-chat").provider, + resolve: () => resolution(profile), + }) + +export const resolver = ProviderResolver.define({ + id: ProviderResolver.make("openai-compatible", "openai-compatible-chat").provider, + resolve: (input) => resolve(input.providerID), +}) + +export const familyResolver = ProviderResolver.define({ + id: ProviderResolver.make("openai-compatible", "openai-compatible-chat").provider, + resolve: (input) => resolveFamily(input.providerID), +}) + +export * as OpenAICompatibleProfiles from "./openai-compatible-profile" diff --git a/packages/llm/src/provider/openai-responses.ts b/packages/llm/src/provider/openai-responses.ts index 160aa898a3..150926510e 100644 --- a/packages/llm/src/provider/openai-responses.ts +++ b/packages/llm/src/provider/openai-responses.ts @@ -383,11 +383,14 @@ export const adapter = Adapter.fromProtocol({ }) export const model = (input: OpenAIResponsesModelInput) => - llmModel({ - ...input, - provider: "openai", - protocol: "openai-responses", - capabilities: input.capabilities ?? capabilities({ tools: { calls: true, streamingInput: true } }), - }) + Adapter.bindModel( + llmModel({ + ...input, + provider: "openai", + protocol: "openai-responses", + capabilities: input.capabilities ?? capabilities({ tools: { calls: true, streamingInput: true } }), + }), + adapter, + ) export * as OpenAIResponses from "./openai-responses" diff --git a/packages/llm/src/provider/openrouter.ts b/packages/llm/src/provider/openrouter.ts index 2351475288..b8541244de 100644 --- a/packages/llm/src/provider/openrouter.ts +++ b/packages/llm/src/provider/openrouter.ts @@ -1,24 +1,25 @@ -import { ProviderResolver } from "../provider-resolver" import { OpenAICompatible, type ModelOptions as OpenAICompatibleModelOptions } from "./openai-compatible" +import { OpenAICompatibleProfiles } from "./openai-compatible-profile" -const baseURL = "https://openrouter.ai/api/v1" +export const profile = OpenAICompatibleProfiles.profiles.openrouter export type ModelOptions = Omit & { readonly baseURL?: string } -export const resolver = ProviderResolver.fixed("openrouter", "openai-compatible-chat", { - baseURL, -}) +export const resolver = OpenAICompatibleProfiles.resolverFor(profile) export const adapters = OpenAICompatible.adapters -export const model = (id: string, options: ModelOptions = {}) => - OpenAICompatible.model(id, { +export const model = (id: string, options: ModelOptions = {}) => { + const baseURL = options.baseURL ?? profile.baseURL + if (!baseURL) throw new Error("OpenRouter requires a baseURL") + return OpenAICompatible.model(id, { ...options, - provider: "openrouter", - baseURL: options.baseURL ?? baseURL, + provider: profile.provider, + baseURL, }) +} export const chat = model diff --git a/packages/llm/src/provider/shared.ts b/packages/llm/src/provider/shared.ts index a59303b969..524f06da37 100644 --- a/packages/llm/src/provider/shared.ts +++ b/packages/llm/src/provider/shared.ts @@ -169,14 +169,14 @@ export const invalidRequest = (message: string) => new InvalidRequestError({ mes /** * Build a `validate` step from a Schema decoder. Replaces the per-adapter - * lambda body `(draft) => decode(draft).pipe(Effect.mapError((e) => + * lambda body `(target) => decode(target).pipe(Effect.mapError((e) => * invalid(e.message)))`. Any decode error is translated into * `InvalidRequestError` carrying the original parse-error message. */ export const validateWith = (decode: (input: I) => Effect.Effect) => - (draft: I) => - decode(draft).pipe(Effect.mapError((error) => invalidRequest(error.message))) + (target: I) => + decode(target).pipe(Effect.mapError((error) => invalidRequest(error.message))) /** * Build an HTTP POST with a JSON body. Sets `content-type: application/json` diff --git a/packages/llm/src/schema.ts b/packages/llm/src/schema.ts index 287c60ec67..fd9cda6844 100644 --- a/packages/llm/src/schema.ts +++ b/packages/llm/src/schema.ts @@ -6,14 +6,7 @@ import { Schema } from "effect" * the runtime registry keys lookups by it. The implementation type itself is * `Protocol` (see `protocol.ts`). */ -export const ProtocolID = Schema.Literals([ - "openai-chat", - "openai-compatible-chat", - "openai-responses", - "anthropic-messages", - "gemini", - "bedrock-converse", -]) +export const ProtocolID = Schema.String export type ProtocolID = Schema.Schema.Type export const ModelID = Schema.String.pipe(Schema.brand("LLM.ModelID")) diff --git a/packages/llm/test/adapter.test.ts b/packages/llm/test/adapter.test.ts index 734fbd2851..61aa82fccc 100644 --- a/packages/llm/test/adapter.test.ts +++ b/packages/llm/test/adapter.test.ts @@ -53,7 +53,7 @@ const mapText = (fn: (text: string) => string) => (request: LLMRequest): LLMRequ const Json = Schema.fromJsonString(Schema.Unknown) const encodeJson = Schema.encodeSync(Json) -type FakeDraft = { +type FakeTarget = { readonly body: string readonly includeUsage?: boolean } @@ -80,10 +80,10 @@ const raiseChunk = (chunk: FakeChunk): import("../src/schema").LLMEvent => ? { type: "request-finish", reason: chunk.reason } : { type: "text-delta", text: chunk.text } -const fake = Adapter.unsafe({ +const fake = Adapter.unsafe({ id: "fake", protocol: "openai-chat", - validate: (draft) => Effect.succeed(draft), + validate: (target) => Effect.succeed(target), prepare: (request) => Effect.succeed({ body: [ @@ -113,7 +113,7 @@ const fake = Adapter.unsafe({ ), }) -const gemini = Adapter.unsafe({ +const gemini = Adapter.unsafe({ ...fake, id: "gemini-fake", protocol: "gemini", @@ -140,7 +140,7 @@ describe("llm adapter", () => { fake.withPatches([ fake.patch("include-usage", { reason: "fake target patch", - apply: (draft) => ({ ...draft, includeUsage: true }), + apply: (target) => ({ ...target, includeUsage: true }), }), ]), ], @@ -172,6 +172,33 @@ describe("llm adapter", () => { }), ) + it.effect("falls back to adapter bound to model", () => + Effect.gen(function* () { + const prepared = yield* LLMClient.make({ adapters: [] }).prepare( + LLM.updateRequest(request, { + model: Adapter.bindModel(updateModel(request.model, { protocol: "gemini" }), gemini), + }), + ) + + expect(prepared.adapter).toBe("gemini-fake") + }), + ) + + it.effect("explicit adapters override provider adapters", () => + Effect.gen(function* () { + const override = Adapter.unsafe({ + ...fake, + id: "fake-override", + prepare: () => Effect.succeed({ body: "override" }), + }) + + const prepared = yield* LLM.make({ providers: [{ adapters: [fake] }], adapters: [override] }).prepare(request) + + expect(prepared.adapter).toBe("fake-override") + expect(prepared.target).toEqual({ body: "override" }) + }), + ) + it.effect("request, prompt, and tool-schema patches run before adapter prepare", () => Effect.gen(function* () { const prepared = yield* LLMClient.make({ diff --git a/packages/llm/test/provider-resolver.test.ts b/packages/llm/test/provider-resolver.test.ts index 17a66dd887..83f2b6bf74 100644 --- a/packages/llm/test/provider-resolver.test.ts +++ b/packages/llm/test/provider-resolver.test.ts @@ -1,5 +1,5 @@ import { describe, expect, test } from "bun:test" -import { Azure, GitHubCopilot, OpenAI, OpenAICompatibleFamily, ProviderResolver } from "../src" +import { Azure, GitHubCopilot, OpenAI, OpenAICompatibleFamily, OpenAICompatibleProfiles, OpenRouter, ProviderResolver } from "../src" describe("provider resolver", () => { test("fixed providers resolve protocol and auth defaults", () => { @@ -30,6 +30,18 @@ describe("provider resolver", () => { baseURL: "https://api.together.xyz/v1", auth: "key", }) + expect(OpenAICompatibleProfiles.resolve("deepseek")).toMatchObject({ + provider: "deepseek", + protocol: "openai-compatible-chat", + baseURL: "https://api.deepseek.com/v1", + auth: "key", + }) + expect(OpenRouter.resolver.resolve(ProviderResolver.input("openai/gpt-4o-mini", "openrouter", {}))).toMatchObject({ + provider: "openrouter", + protocol: "openai-compatible-chat", + baseURL: "https://openrouter.ai/api/v1", + auth: "key", + }) }) test("Azure resolves resource URLs and API-version query params", () => { diff --git a/packages/llm/test/schema.test.ts b/packages/llm/test/schema.test.ts index d80acfa5e9..cf0eda5eec 100644 --- a/packages/llm/test/schema.test.ts +++ b/packages/llm/test/schema.test.ts @@ -35,16 +35,16 @@ describe("llm schema", () => { expect(decoded.messages[0]?.content[0]?.type).toBe("text") }) - test("rejects invalid protocol", () => { - expect(() => - Schema.decodeUnknownSync(LLMRequest)({ - model: { ...model, protocol: "bogus" }, - system: [], - messages: [], - tools: [], - generation: {}, - }), - ).toThrow() + test("accepts custom protocol ids", () => { + const decoded = Schema.decodeUnknownSync(LLMRequest)({ + model: { ...model, protocol: "custom-protocol" }, + system: [], + messages: [], + tools: [], + generation: {}, + }) + + expect(decoded.model.protocol).toBe("custom-protocol") }) test("rejects invalid event type", () => { diff --git a/packages/opencode/src/session/llm.ts b/packages/opencode/src/session/llm.ts index b680232e6f..9cc1f4bce3 100644 --- a/packages/opencode/src/session/llm.ts +++ b/packages/opencode/src/session/llm.ts @@ -489,7 +489,8 @@ const live: Layer.Layer< // (the AI SDK `messages` array isn't enough — the LLM-native bridge // needs the typed parts). // - The bridge can route the model to one of the protocols listed in - // `NATIVE_PROTOCOLS` (today: Anthropic only). + // `NATIVE_PROTOCOLS`. The adapter registry is broader than this + // allowlist so we can enable providers incrementally. // - If tools are present, the caller supplied a native tool definition // for every AI SDK tool key so the native path can dispatch them. //