Mirror_Repos/anomalyco_opencode
2
0
Fork 0
Code Issues Pull Requests Actions 70 Packages Projects Releases Wiki Activity

refactor(llm): collapse client adapter injection

Browse Source
This commit is contained in:
Kit Langton
2026-05-06 11:08:31 -04:00
parent 1808fe8628
commit 7b4f436fc2
29 changed files with 266 additions and 377 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/llm/AGENTS.md
+8 -8
View File
@@ -30,12 +30,12 @@ const request = LLM.request({
prompt: "Say hello.",
})
const response = yield* LLMClient.make().generate(request)
const response = yield* LLMClient.generate(request)
```
`LLM.request(...)` builds an `LLMRequest`. `LLMClient.make(...)` selects an adapter from the model binding or explicit registry by `request.model.adapter`, prepares a typed provider payload, asks the adapter for a real `HttpClientRequest.HttpClientRequest`, sends it through `RequestExecutor.Service`, parses the provider stream into common `LLMEvent`s, and finally returns an `LLMResponse`.
`LLM.request(...)` builds an `LLMRequest`. `LLMClient.generate(...)` selects a registered adapter by `request.model.adapter`, prepares a typed provider payload, asks the adapter for a real `HttpClientRequest.HttpClientRequest`, sends it through `RequestExecutor.Service`, parses the provider stream into common `LLMEvent`s, and finally returns an `LLMResponse`.
Use `LLMClient.make(...).stream(request)` when callers want incremental `LLMEvent`s. Use `LLMClient.make(...).generate(request)` when callers want those same events collected into an `LLMResponse`. Use `LLMClient.make(...).prepare<Payload>(request)` to compile a request through the adapter pipeline without sending it — the optional `Payload` type argument narrows `.payload` to the adapter's native shape (e.g. `prepare<OpenAIChatPayload>(...)` returns a `PreparedRequestOf<OpenAIChatPayload>`). The runtime payload is identical; the generic is a type-level assertion.
Use `LLMClient.stream(request)` when callers want incremental `LLMEvent`s. Use `LLMClient.generate(request)` when callers want those same events collected into an `LLMResponse`. Use `LLMClient.prepare<Payload>(request)` to compile a request through the adapter pipeline without sending it — the optional `Payload` type argument narrows `.payload` to the adapter's native shape (e.g. `prepare<OpenAIChatPayload>(...)` returns a `PreparedRequestOf<OpenAIChatPayload>`). The runtime payload is identical; the generic is a type-level assertion.
Filter or narrow `LLMEvent` streams with `LLMEvent.is.*` (camelCase guards, e.g. `events.filter(LLMEvent.is.toolCall)`). The kebab-case `LLMEvent.guards["tool-call"]` form also works but prefer `is.*` in new code.
@@ -73,7 +73,7 @@ packages/llm/src/
llm.ts // request constructors and convenience helpers
adapter/
index.ts // @opencode-ai/llm/adapter advanced barrel
client.ts // Adapter.make + LLMClient.make
client.ts // Adapter.make + LLMClient.prepare/stream/generate
executor.ts // RequestExecutor service + transport error mapping
protocol.ts // Protocol type + Protocol.define
endpoint.ts // Endpoint type + Endpoint.baseURL
@@ -131,7 +131,7 @@ Adapters lower this into provider-native assistant tool-call messages and tool-r
### Tool runtime
`ToolRuntime.run(client, options)` orchestrates the tool loop with full type safety:
`ToolRuntime.run(options)` orchestrates the tool loop with full type safety:
```ts
const get_weather = tool({
@@ -147,7 +147,7 @@ const get_weather = tool({
}),
})
const events = yield* ToolRuntime.run(client, {
const events = yield* ToolRuntime.run({
request,
tools: { get_weather, get_time, ... },
maxSteps: 10,
@@ -231,7 +231,7 @@ Do not blanket re-record an entire test file when adding one cassette. `RECORD=t
### Completed Foundation
- [x] Add an adapter registry so `LLMClient.make(...)` can choose an adapter by provider/protocol instead of requiring a single adapter.
- [x] Add an adapter registry so `LLMClient` can choose an adapter by provider/protocol instead of requiring a single adapter.
- [x] Add request/response convenience helpers where callsites still expose schema internals, but keep constructors returning canonical Schema class instances.
- [x] Expand OpenAI Chat support for assistant tool-call messages followed by tool-result messages.
- [x] Add OpenAI Chat recorded tests for tool-result follow-up and usage chunks.
@@ -276,7 +276,7 @@ Do not blanket re-record an entire test file when adding one cassette. `RECORD=t
- [x] Add a native event bridge that maps `LLMEvent` streams into the existing `SessionProcessor` event contract without creating a second processor.
- [ ] Extract runtime-neutral OpenCode tool resolution from `SessionPrompt.resolveTools`, then build both existing-stream and native `@opencode-ai/llm` tool adapters from the same resolved shape.
- [ ] Map `Permission.RejectedError`, `Permission.CorrectedError`, validation failures, thrown tool failures, and aborts into model-visible native tool error/results.
- [ ] Wire a native stream producer behind an explicit local flag and provider allowlist; the producer should consume `nativeMessages`, call `LLMNative.request(...)`, stream through `LLMClient.make(...)`, and feed `LLMNativeEvents.mapper()` into `SessionProcessor`.
- [ ] Wire a native stream producer behind an explicit local flag and provider allowlist; the producer should consume `nativeMessages`, call `LLMNative.request(...)`, stream through `LLMClient.stream(...)`, and feed `LLMNativeEvents.mapper()` into `SessionProcessor`.
- [ ] Add end-to-end native stream tests through the actual session loop for text, reasoning, tool-call streaming, tool success, rejected permission, corrected permission, thrown tool error, abort, and provider-executed tool history.
- [ ] Dogfood native streaming with the flag enabled for OpenAI first, then Anthropic, Gemini, OpenAI-compatible providers, Bedrock, and Copilot provider-by-provider.
- [ ] Flip native streaming to default only after request parity, stream parity, tool execution, typecheck, focused provider tests, recorded cassettes, and manual dogfood pass for the enabled provider set.
packages/llm/TOUR.md
+5 -8
View File
@@ -124,7 +124,7 @@ The runtime pipeline is concentrated in [`src/adapter/client.ts`](./src/adapter/
The important functions are:
- `Adapter.model`, which binds a user-facing model helper to the adapter that can run it.
- `LLMClient.make`, which selects an adapter, builds the payload, sends HTTP, and parses the response.
- `LLMClient`, which selects a registered adapter, builds the payload, sends HTTP, and parses the response.
- `Adapter.make`, which composes protocol semantics with endpoint, auth, and framing.
At runtime, the flow is easier to read as a sequence of values. There are two levels to keep separate:
@@ -185,19 +185,17 @@ const request: LLMRequest = LLM.request(input)
// The caller hands that request to the client and chooses one exit path:
// inspect the compiled request, stream events, or collect a final response.
const client: LLMClient = LLMClient.make()
// Alternative A: compile without sending HTTP. Useful for request-shape tests.
// LLMRequest -> PreparedRequestOf<Payload>
const prepared: PreparedRequestOf<Payload> = client.prepare<Payload>(request)
const prepared: PreparedRequestOf<Payload> = LLMClient.prepare<Payload>(request)
// Alternative B: send HTTP and expose normalized stream events.
// LLMRequest -> Stream<LLMEvent>
const streamed: Stream.Stream<LLMEvent, LLMError> = client.stream(request)
const streamed: Stream.Stream<LLMEvent, LLMError> = LLMClient.stream(request)
// Alternative C: send HTTP and collect those same events into one response.
// LLMRequest -> LLMResponse
const generated: LLMResponse = client.generate(request)
const generated: LLMResponse = LLMClient.generate(request)
// -----------------------------------------------------------------------------
// Stage 3: Client Compiles The Request
@@ -205,8 +203,7 @@ const generated: LLMResponse = client.generate(request)
// Internally, all three alternatives start by compiling the request. The client
// first resolves model defaults plus request overrides, then selects the
// runnable adapter from the model binding or an explicit registry keyed by
// `request.model.adapter`.
// runnable adapter from the registry keyed by `request.model.adapter`.
const resolvedRequest: LLMRequest = resolveModelAndCallOptions(request)
const adapter: Adapter<Payload> = resolveAdapter(request.model)
packages/llm/example/tutorial.ts
+3 -3
View File
@@ -172,7 +172,7 @@ const FakeEcho = {
// payload conversion, validation, endpoint, auth, and HTTP construction without
// sending anything over the network.
const inspectFakeProvider = Effect.gen(function* () {
const prepared = yield* LLMClient.make().prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
model: FakeEcho.model("tiny-echo"),
prompt: "Show me the provider pipeline.",
@@ -190,9 +190,9 @@ const inspectFakeProvider = Effect.gen(function* () {
const program = Effect.gen(function* () {
// yield* generateOnce
// yield* inspectFakeProvider
// yield* LLMClient.make().prepare(rawOverlayExample).pipe(Effect.andThen((prepared) => Effect.sync(() => console.log(prepared.payload))))
// yield* LLMClient.prepare(rawOverlayExample).pipe(Effect.andThen((prepared) => Effect.sync(() => console.log(prepared.payload))))
// yield* streamText
yield* streamWithTools
}).pipe(Effect.provide(Layer.mergeAll(LLM.layer(), RequestExecutor.defaultLayer)))
}).pipe(Effect.provide(LLM.layer.pipe(Layer.provide(RequestExecutor.defaultLayer))))
Effect.runPromise(program)
packages/llm/src/adapter/client.ts
+61 -77
View File
@@ -52,20 +52,17 @@ export interface Adapter<Payload> {
) => Stream.Stream<LLMEvent, LLMError>
}
export type AdapterInput<Payload> = Adapter<Payload>
export interface AdapterDefinition<Payload> extends Adapter<Payload> {}
// Adapter registries intentionally erase payload generics after the typed
// adapter is constructed. This keeps normal call sites on `OpenAIChat.adapter`
// instead of leaking a separate runtime-adapter wrapper.
// Adapter registries intentionally erase payload generics after construction.
// Normal call sites use `OpenAIChat.adapter`; callers only need payload types
// when preparing a request with a protocol-specific type assertion.
// oxlint-disable-next-line typescript-eslint/no-explicit-any
export type AnyAdapter = AdapterDefinition<any>
export type AnyAdapter = Adapter<any>
const adapterRegistry = new Map<string, AnyAdapter>()
// The first adapter registered for an id is the package default. Tests and
// advanced callers can still override per-client via `LLMClient.make({ adapters })`.
// The first adapter registered for an id is the package default. Adapter lookup
// is intentionally global: model refs name an adapter id, and importing the
// provider/protocol/custom-adapter module registers the runnable implementation.
const register = <Adapter extends AnyAdapter>(adapter: Adapter): Adapter => {
if (!adapterRegistry.has(adapter.id)) adapterRegistry.set(adapter.id, adapter)
return adapter
@@ -202,10 +199,6 @@ export interface LLMClient {
readonly generate: (request: LLMRequest) => Effect.Effect<LLMResponse, LLMError, RequestExecutor.Service>
}
export interface ClientOptions {
readonly adapters?: ReadonlyArray<AnyAdapter>
}
const noAdapter = (model: ModelRef) =>
new NoAdapterError({ adapter: model.adapter, protocol: model.protocol, provider: model.provider, model: model.id })
@@ -254,7 +247,7 @@ export interface MakeInput<Payload, Frame, Chunk, State> {
*/
export function make<Payload, Frame, Chunk, State>(
input: MakeInput<Payload, Frame, Chunk, State>,
): AdapterDefinition<Payload> {
): Adapter<Payload> {
const auth = input.auth ?? authBearer
const protocol = input.protocol
const encodePayload = Schema.encodeSync(Schema.fromJsonString(protocol.payload))
@@ -321,77 +314,68 @@ export function make<Payload, Frame, Chunk, State>(
})
}
/**
* Build the lower-level runtime. `compile` is the important boundary: it turns
* a common `LLMRequest` into a validated provider payload plus HTTP request,
* but does not execute transport.
*/
const makeClient = (options: ClientOptions = {}): LLMClient => {
const adapters = new Map((options.adapters ?? []).map((adapter) => [adapter.id, adapter] as const))
// `compile` is the important boundary: it turns a common `LLMRequest` into a
// validated provider payload plus HTTP request, but does not execute transport.
const compile = Effect.fn("LLM.compile")(function* (request: LLMRequest) {
const resolved = resolveRequestOptions(request)
const adapter = registeredAdapter(resolved.model.adapter)
if (!adapter) return yield* noAdapter(resolved.model)
const compile = Effect.fn("LLM.compile")(function* (request: LLMRequest) {
const resolved = resolveRequestOptions(request)
const adapter = adapters.get(resolved.model.adapter) ?? registeredAdapter(resolved.model.adapter)
if (!adapter) return yield* noAdapter(resolved.model)
const payload = yield* adapter.toPayload(resolved).pipe(
Effect.flatMap(ProviderShared.validateWith(Schema.decodeUnknownEffect(adapter.payloadSchema))),
)
const http = yield* adapter.toHttp(payload, {
request: resolved,
})
return {
request: resolved,
adapter,
payload,
http,
}
const payload = yield* adapter.toPayload(resolved).pipe(
Effect.flatMap(ProviderShared.validateWith(Schema.decodeUnknownEffect(adapter.payloadSchema))),
)
const http = yield* adapter.toHttp(payload, {
request: resolved,
})
const prepare = Effect.fn("LLMClient.prepare")(function* (request: LLMRequest) {
const compiled = yield* compile(request)
return {
request: resolved,
adapter,
payload,
http,
}
})
return new PreparedRequest({
id: compiled.request.id ?? "request",
adapter: compiled.adapter.id,
model: compiled.request.model,
payload: compiled.payload,
})
const prepare = Effect.fn("LLMClient.prepare")(function* (request: LLMRequest) {
const compiled = yield* compile(request)
return new PreparedRequest({
id: compiled.request.id ?? "request",
adapter: compiled.adapter.id,
model: compiled.request.model,
payload: compiled.payload,
})
})
const stream = (request: LLMRequest) =>
Stream.unwrap(
Effect.gen(function* () {
const compiled = yield* compile(request)
const executor = yield* RequestExecutor.Service
const response = yield* executor.execute(compiled.http)
const stream = (request: LLMRequest) =>
Stream.unwrap(
Effect.gen(function* () {
const compiled = yield* compile(request)
const executor = yield* RequestExecutor.Service
const response = yield* executor.execute(compiled.http)
return compiled.adapter.parse(response, { request: compiled.request })
}),
)
return compiled.adapter.parse(response, { request: compiled.request })
}),
)
const generate = Effect.fn("LLM.generate")(function* (request: LLMRequest) {
return new LLMResponse(
yield* stream(request).pipe(
Stream.runFold(
() => ({ events: [] as LLMEvent[], usage: undefined as LLMResponse["usage"] }),
(acc, event) => {
acc.events.push(event)
if ("usage" in event && event.usage !== undefined) acc.usage = event.usage
return acc
},
),
const generate = Effect.fn("LLM.generate")(function* (request: LLMRequest) {
return new LLMResponse(
yield* stream(request).pipe(
Stream.runFold(
() => ({ events: [] as LLMEvent[], usage: undefined as LLMResponse["usage"] }),
(acc, event) => {
acc.events.push(event)
if ("usage" in event && event.usage !== undefined) acc.usage = event.usage
return acc
},
),
)
})
// The runtime always emits a `PreparedRequest` (payload: unknown). Callers
// who supply a `Payload` type argument assert the shape they expect from
// their adapter; the cast hands them a typed view of the same payload.
return { prepare: prepare as LLMClient["prepare"], stream, generate }
}
),
)
})
export const Adapter = { make, model } as const
export const LLMClient = { make: makeClient }
// The runtime always emits a `PreparedRequest` (payload: unknown). Callers who
// supply a `Payload` type argument assert the shape they expect from their
// adapter; the cast hands them a typed view of the same payload.
export const LLMClient: LLMClient = { prepare: prepare as LLMClient["prepare"], stream, generate }
packages/llm/src/adapter/index.ts
-3
View File
@@ -1,14 +1,11 @@
export { Adapter, LLMClient, modelCapabilities, modelLimits, modelRef } from "./client"
export type {
Adapter as AdapterShape,
AdapterDefinition,
AdapterInput,
AdapterModelDefaults,
AdapterModelInput,
AdapterRoutedModelDefaults,
AdapterRoutedModelInput,
AnyAdapter,
ClientOptions,
HttpContext,
LLMClient as LLMClientShape,
ModelCapabilitiesInput,
packages/llm/src/index.ts
-1
View File
@@ -2,7 +2,6 @@ export { LLMClient, modelCapabilities, modelLimits, modelRef } from "./adapter/c
export type {
AdapterModelInput,
AdapterRoutedModelInput,
ClientOptions,
LLMClient as LLMClientShape,
ModelCapabilitiesInput,
ModelRefInput,
packages/llm/src/llm.ts
+21 -15
View File
@@ -7,7 +7,7 @@ import {
type ModelCapabilitiesInput,
type ModelRefInput,
} from "./adapter/client"
import type { RequestExecutor } from "./adapter/executor"
import { RequestExecutor } from "./adapter/executor"
import { type Tools } from "./tool"
import { ToolRuntime, type RunOptions } from "./tool-runtime"
import {
@@ -31,31 +31,37 @@ import type { LLMError } from "./schema"
export type StreamWithToolsInput<T extends Tools> = Omit<RequestInput, "tools"> & Omit<RunOptions<T>, "request">
export interface Runtime {
readonly stream: (input: LLMRequest | RequestInput) => Stream.Stream<LLMEvent, LLMError, RequestExecutor.Service>
readonly generate: (input: LLMRequest | RequestInput) => Effect.Effect<LLMResponse, LLMError, RequestExecutor.Service>
export interface Interface {
readonly stream: (input: LLMRequest | RequestInput) => Stream.Stream<LLMEvent, LLMError>
readonly generate: (input: LLMRequest | RequestInput) => Effect.Effect<LLMResponse, LLMError>
readonly streamWithTools: <T extends Tools>(
input: StreamWithToolsInput<T>,
) => Stream.Stream<LLMEvent, LLMError, RequestExecutor.Service>
) => Stream.Stream<LLMEvent, LLMError>
}
export class Service extends Context.Service<Service, Runtime>()("@opencode/LLM") {}
export class Service extends Context.Service<Service, Interface>()("@opencode/LLM") {}
const requestOf = (input: LLMRequest | RequestInput) => (input instanceof LLMRequest ? input : request(input))
export const make = (): Runtime => {
const client = LLMClient.make()
return {
stream: (input) => client.stream(requestOf(input)),
generate: (input) => client.generate(requestOf(input)),
export const make = (executor: RequestExecutor.Interface): Interface => ({
stream: (input) =>
LLMClient.stream(requestOf(input)).pipe(Stream.provideService(RequestExecutor.Service, executor)),
generate: (input) =>
LLMClient.generate(requestOf(input)).pipe(Effect.provideService(RequestExecutor.Service, executor)),
streamWithTools: (input) => {
const { maxSteps, concurrency, stopWhen, tools, ...rest } = input
return ToolRuntime.run(client, { request: request(rest), tools, maxSteps, concurrency, stopWhen })
return ToolRuntime.run({ request: request(rest), tools, maxSteps, concurrency, stopWhen }).pipe(
Stream.provideService(RequestExecutor.Service, executor),
)
},
}
}
})
export const layer = (): Layer.Layer<Service> => Layer.succeed(Service, Service.of(make()))
export const layer: Layer.Layer<Service, never, RequestExecutor.Service> = Layer.effect(
Service,
Effect.gen(function* () {
return Service.of(make(yield* RequestExecutor.Service))
}),
)
export const stream = (input: LLMRequest | RequestInput) =>
Stream.unwrap(
packages/llm/src/tool-runtime.ts
+3 -6
View File
@@ -1,6 +1,6 @@
import { Effect, Stream } from "effect"
import type { Concurrency } from "effect/Types"
import type { LLMClient } from "./adapter/client"
import { LLMClient } from "./adapter/client"
import type { RequestExecutor } from "./adapter/executor"
import {
type ContentPart,
@@ -56,10 +56,7 @@ export interface RunOptions<T extends Tools> {
* Tool handler dependencies are closed over at tool definition time, so the
* runtime's only environment requirement is the `RequestExecutor.Service`.
*/
export const run = <T extends Tools>(
client: LLMClient,
options: RunOptions<T>,
): Stream.Stream<LLMEvent, LLMError, RequestExecutor.Service> => {
export const run = <T extends Tools>(options: RunOptions<T>): Stream.Stream<LLMEvent, LLMError, RequestExecutor.Service> => {
const maxSteps = options.maxSteps ?? 10
const concurrency = options.concurrency ?? 10
const tools = options.tools as Tools
@@ -80,7 +77,7 @@ export const run = <T extends Tools>(
Effect.gen(function* () {
const state: StepState = { assistantContent: [], toolCalls: [], finishReason: undefined }
const modelStream = client.stream(request).pipe(
const modelStream = LLMClient.stream(request).pipe(
Stream.tap((event) => Effect.sync(() => accumulate(state, event))),
)
packages/llm/test/adapter.test.ts
+5 -23
View File
@@ -101,7 +101,7 @@ const it = testEffect(echoLayer)
describe("llm adapter", () => {
it.effect("stream and generate use the adapter pipeline", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [fake] })
const llm = LLMClient
const events = Array.from(yield* llm.stream(request).pipe(Stream.runCollect))
const response = yield* llm.generate(request)
@@ -112,7 +112,7 @@ describe("llm adapter", () => {
it.effect("selects adapters by request adapter", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [fake, gemini] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.updateRequest(request, { model: updateModel(request.model, { adapter: "gemini-fake" }) }),
)
@@ -122,7 +122,7 @@ describe("llm adapter", () => {
it.effect("uses registered adapters by model adapter id", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.updateRequest(request, { model: updateModel(request.model, { adapter: "gemini-fake" }) }),
)
@@ -147,24 +147,6 @@ describe("llm adapter", () => {
}),
)
it.effect("explicit adapters override provider adapters", () =>
Effect.gen(function* () {
const override = Adapter.make({
id: "fake",
protocol: Protocol.define({
...fakeProtocol,
toPayload: () => Effect.succeed({ body: "override" }),
}),
endpoint: Endpoint.baseURL({ default: "https://fake.local", path: "/chat" }),
framing: fakeFraming,
})
const response = yield* LLMClient.make({ adapters: [override] }).generate(request)
expect(response.text).toBe('echo:{"body":"override"}')
}),
)
it.effect("keeps the first registered adapter as the default", () =>
Effect.gen(function* () {
Adapter.make({
@@ -177,7 +159,7 @@ describe("llm adapter", () => {
framing: fakeFraming,
})
const response = yield* LLMClient.make().generate(request)
const response = yield* LLMClient.generate(request)
expect(response.text).toBe('echo:{"body":"hello"}')
}),
@@ -185,7 +167,7 @@ describe("llm adapter", () => {
it.effect("rejects missing adapter", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [fake] })
const error = yield* LLMClient
.prepare(
LLM.updateRequest(request, { model: updateModel(request.model, { adapter: "missing" }) }),
)
packages/llm/test/exports.test.ts
+1 -1
View File
@@ -9,7 +9,7 @@ import * as AnthropicMessages from "@opencode-ai/llm/protocols/anthropic-message
describe("public exports", () => {
test("root exposes app-facing runtime APIs", () => {
expect(LLM.generate).toBeFunction()
expect(LLMClient.make).toBeFunction()
expect(LLMClient.generate).toBeFunction()
})
test("adapter barrel exposes adapter-authoring APIs", () => {
packages/llm/test/provider/anthropic-messages.recorded.test.ts
+2 -2
View File
@@ -31,7 +31,7 @@ const recorded = recordedTests({
requires: ["ANTHROPIC_API_KEY"],
options: { requestHeaders: ["content-type", "anthropic-version"] },
})
const anthropic = LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const anthropic = LLMClient
const malformedToolOrderRequest = LLM.request({
id: "recorded_anthropic_malformed_tool_order",
@@ -72,7 +72,7 @@ describe("Anthropic Messages recorded", () => {
recorded.effect.with("claude opus 4.7 drives a tool loop", { tags: ["tool", "tool-loop", "golden", "flagship"] }, () =>
Effect.gen(function* () {
expectWeatherToolLoop(yield* runWeatherToolLoop(anthropic, flagshipToolLoopRequest))
expectWeatherToolLoop(yield* runWeatherToolLoop(flagshipToolLoopRequest))
}),
)
packages/llm/test/provider/anthropic-messages.test.ts
+11 -11
View File
@@ -26,7 +26,7 @@ const it = testEffect(Layer.empty)
describe("Anthropic Messages adapter", () => {
it.effect("prepares Anthropic Messages target", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] }).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toEqual({
model: "claude-sonnet-4-5",
@@ -41,7 +41,7 @@ describe("Anthropic Messages adapter", () => {
it.effect("prepares tool call and tool result messages", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_tool_result",
model,
@@ -80,7 +80,7 @@ describe("Anthropic Messages adapter", () => {
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 2 } },
{ type: "message_stop" },
)
const response = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(body)))
@@ -106,7 +106,7 @@ describe("Anthropic Messages adapter", () => {
{ type: "content_block_stop", index: 0 },
{ type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 1 } },
)
const response = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
@@ -130,7 +130,7 @@ describe("Anthropic Messages adapter", () => {
it.effect("emits provider-error events for mid-stream provider errors", () =>
Effect.gen(function* () {
const response = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const response = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
@@ -144,7 +144,7 @@ describe("Anthropic Messages adapter", () => {
it.effect("fails HTTP provider errors before stream parsing", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const error = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
@@ -184,7 +184,7 @@ describe("Anthropic Messages adapter", () => {
{ type: "content_block_stop", index: 2 },
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 8 } },
)
const response = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "web_search", description: "Web search", inputSchema: { type: "object" } }],
@@ -232,7 +232,7 @@ describe("Anthropic Messages adapter", () => {
{ type: "content_block_stop", index: 1 },
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } },
)
const response = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "web_search", description: "Web search", inputSchema: { type: "object" } }],
@@ -253,7 +253,7 @@ describe("Anthropic Messages adapter", () => {
it.effect("round-trips provider-executed assistant content into server tool blocks", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_round_trip",
model,
@@ -304,7 +304,7 @@ describe("Anthropic Messages adapter", () => {
it.effect("rejects round-trip for unknown server tool names", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_unknown_server_tool",
@@ -330,7 +330,7 @@ describe("Anthropic Messages adapter", () => {
it.effect("rejects unsupported user media content", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] })
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_media",
packages/llm/test/provider/bedrock-converse.test.ts
+20 -20
View File
@@ -64,7 +64,7 @@ const it = testEffect(Layer.empty)
describe("Bedrock Converse adapter", () => {
it.effect("prepares Converse target with system, inference config, and messages", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(baseRequest)
const prepared = yield* LLMClient.prepare(baseRequest)
expect(prepared.payload).toEqual({
modelId: "anthropic.claude-3-5-sonnet-20240620-v1:0",
@@ -77,7 +77,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("prepares tool config with toolSpec and toolChoice", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.updateRequest(baseRequest, {
tools: [
{
@@ -111,7 +111,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("lowers assistant tool-call + tool-result message history", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_history",
model,
@@ -157,7 +157,7 @@ describe("Bedrock Converse adapter", () => {
["messageStop", { stopReason: "end_turn" }],
["metadata", { usage: { inputTokens: 5, outputTokens: 2, totalTokens: 7 } }],
)
const response = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] })
const response = yield* LLMClient
.generate(baseRequest)
.pipe(Effect.provide(fixedBytes(body)))
@@ -192,7 +192,7 @@ describe("Bedrock Converse adapter", () => {
["contentBlockStop", { contentBlockIndex: 0 }],
["messageStop", { stopReason: "tool_use" }],
)
const response = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] })
const response = yield* LLMClient
.generate(
LLM.updateRequest(baseRequest, {
tools: [{ name: "lookup", description: "Lookup", inputSchema: { type: "object" } }],
@@ -223,7 +223,7 @@ describe("Bedrock Converse adapter", () => {
["contentBlockStop", { contentBlockIndex: 0 }],
["messageStop", { stopReason: "end_turn" }],
)
const response = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] })
const response = yield* LLMClient
.generate(baseRequest)
.pipe(Effect.provide(fixedBytes(body)))
@@ -237,7 +237,7 @@ describe("Bedrock Converse adapter", () => {
["messageStart", { role: "assistant" }],
["throttlingException", { message: "Slow down" }],
)
const response = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] })
const response = yield* LLMClient
.generate(baseRequest)
.pipe(Effect.provide(fixedBytes(body)))
@@ -255,7 +255,7 @@ describe("Bedrock Converse adapter", () => {
id: "anthropic.claude-3-5-sonnet-20240620-v1:0",
baseURL: "https://bedrock-runtime.test",
})
const error = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] })
const error = yield* LLMClient
.generate(LLM.updateRequest(baseRequest, { model: unsignedModel }))
.pipe(Effect.provide(fixedBytes(eventStreamBody(["messageStop", { stopReason: "end_turn" }]))), Effect.flip)
@@ -274,7 +274,7 @@ describe("Bedrock Converse adapter", () => {
secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
},
})
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.updateRequest(baseRequest, { model: signed }),
)
@@ -291,7 +291,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("emits cachePoint markers after system, user-text, and assistant-text with cache hints", () =>
Effect.gen(function* () {
const cache = new CacheHint({ type: "ephemeral" })
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_cache",
model,
@@ -323,7 +323,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("does not emit cachePoint when no cache hint is set", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(baseRequest)
const prepared = yield* LLMClient.prepare(baseRequest)
expect(prepared.payload).toMatchObject({
system: [{ text: "You are concise." }],
messages: [{ role: "user", content: [{ text: "Say hello." }] }],
@@ -333,7 +333,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("lowers image media into Bedrock image blocks", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_image",
model,
@@ -369,7 +369,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("base64-encodes Uint8Array image bytes", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_image_bytes",
model,
@@ -395,7 +395,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("lowers document media into Bedrock document blocks with format and name", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_doc",
model,
@@ -426,7 +426,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("rejects unsupported image media types", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] })
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_bad_image",
@@ -442,7 +442,7 @@ describe("Bedrock Converse adapter", () => {
it.effect("rejects unsupported document media types", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [BedrockConverse.adapter] })
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_bad_doc",
@@ -494,7 +494,7 @@ const recorded = recordedTests({
describe("Bedrock Converse recorded", () => {
recorded.effect("streams text", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [BedrockConverse.adapter] })
const llm = LLMClient
const response = yield* llm.generate(
LLM.request({
id: "recorded_bedrock_text",
@@ -514,7 +514,7 @@ describe("Bedrock Converse recorded", () => {
recorded.effect.with("streams a tool call", { tags: ["tool"] }, () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [BedrockConverse.adapter] })
const llm = LLMClient
const response = yield* llm.generate(
LLM.request({
id: "recorded_bedrock_tool_call",
@@ -536,8 +536,8 @@ describe("Bedrock Converse recorded", () => {
recorded.effect.with("drives a tool loop", { tags: ["tool", "tool-loop", "golden"] }, () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [BedrockConverse.adapter] })
expectWeatherToolLoop(yield* runWeatherToolLoop(llm, weatherToolLoopRequest({
const llm = LLMClient
expectWeatherToolLoop(yield* runWeatherToolLoop(weatherToolLoopRequest({
id: "recorded_bedrock_tool_loop",
model: recordedModel(),
})))
packages/llm/test/provider/gemini.recorded.test.ts
+1 -1
View File
@@ -20,7 +20,7 @@ const recorded = recordedTests({
protocol: "gemini",
requires: ["GOOGLE_GENERATIVE_AI_API_KEY"],
})
const gemini = LLMClient.make({ adapters: [Gemini.adapter] })
const gemini = LLMClient
describe("Gemini recorded", () => {
recorded.effect("streams text", () =>
packages/llm/test/provider/gemini.test.ts
+12 -12
View File
@@ -26,7 +26,7 @@ const it = testEffect(Layer.empty)
describe("Gemini adapter", () => {
it.effect("prepares Gemini target", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [Gemini.adapter] }).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toEqual({
contents: [{ role: "user", parts: [{ text: "Say hello." }] }],
@@ -38,7 +38,7 @@ describe("Gemini adapter", () => {
it.effect("prepares multimodal user input and tool history", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [Gemini.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_tool_result",
model,
@@ -91,7 +91,7 @@ describe("Gemini adapter", () => {
it.effect("omits tools when tool choice is none", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [Gemini.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_no_tools",
model,
@@ -109,7 +109,7 @@ describe("Gemini adapter", () => {
it.effect("sanitizes integer enums, dangling required, untyped arrays, and scalar object keys", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [Gemini.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_schema_patch",
model,
@@ -177,7 +177,7 @@ describe("Gemini adapter", () => {
},
},
)
const response = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(body)))
@@ -230,7 +230,7 @@ describe("Gemini adapter", () => {
usageMetadata: { promptTokenCount: 5, candidatesTokenCount: 1 },
},
)
const response = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
@@ -266,7 +266,7 @@ describe("Gemini adapter", () => {
}],
},
)
const response = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
@@ -284,14 +284,14 @@ describe("Gemini adapter", () => {
it.effect("maps length and content-filter finish reasons", () =>
Effect.gen(function* () {
const length = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const length = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
fixedResponse(sseEvents({ candidates: [{ content: { role: "model", parts: [] }, finishReason: "MAX_TOKENS" }] })),
),
)
const filtered = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const filtered = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
@@ -306,7 +306,7 @@ describe("Gemini adapter", () => {
it.effect("leaves total usage undefined when component counts are missing", () =>
Effect.gen(function* () {
const response = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(sseEvents({ usageMetadata: { thoughtsTokenCount: 1 } }))))
@@ -317,7 +317,7 @@ describe("Gemini adapter", () => {
it.effect("fails invalid stream chunks", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const error = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(fixedResponse(sseRaw("data: {not json}"))),
@@ -331,7 +331,7 @@ describe("Gemini adapter", () => {
it.effect("rejects unsupported assistant media content", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make({ adapters: [Gemini.adapter] })
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_media",
packages/llm/test/provider/openai-chat-tool-loop.recorded.test.ts
+1 -4
View File
@@ -1,7 +1,6 @@
import { describe, expect } from "bun:test"
import { Effect, Stream } from "effect"
import { LLM } from "../../src"
import { LLMClient } from "../../src/adapter"
import * as OpenAIChat from "../../src/protocols/openai-chat"
import { ToolRuntime } from "../../src/tool-runtime"
import { eventSummary, weatherRuntimeTool } from "../recorded-scenarios"
@@ -32,13 +31,11 @@ const recorded = recordedTests({
protocol: "openai-chat",
requires: ["OPENAI_API_KEY"],
})
const openai = LLMClient.make({ adapters: [OpenAIChat.adapter] })
describe("OpenAI Chat tool-loop recorded", () => {
recorded.effect.with("drives a tool loop end-to-end", { tags: ["tool", "tool-loop"] }, () =>
Effect.gen(function* () {
const events = Array.from(
yield* ToolRuntime.run(openai, { request, tools: { get_weather: weatherRuntimeTool } }).pipe(Stream.runCollect),
yield* ToolRuntime.run({ request, tools: { get_weather: weatherRuntimeTool } }).pipe(Stream.runCollect),
)
expect(LLM.outputText({ events })).toContain("Paris")
packages/llm/test/provider/openai-chat.recorded.test.ts
+1 -1
View File
@@ -36,7 +36,7 @@ const recorded = recordedTests({
protocol: "openai-chat",
requires: ["OPENAI_API_KEY"],
})
const openai = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const openai = LLMClient
describe("OpenAI Chat recorded", () => {
recorded.effect("streams text", () =>
packages/llm/test/provider/openai-chat.test.ts
+15 -15
View File
@@ -37,7 +37,7 @@ describe("OpenAI Chat adapter", () => {
// Pass the OpenAIChat payload type so `prepared.payload` is statically
// typed to the adapter's native shape — the assertions below read field
// names without `unknown` casts.
const prepared = yield* LLMClient.make().prepare<OpenAIChat.OpenAIChatPayload>(request)
const prepared = yield* LLMClient.prepare<OpenAIChat.OpenAIChatPayload>(request)
const _typed: { readonly model: string; readonly stream: true } = prepared.payload
expect(prepared.payload).toEqual({
@@ -56,7 +56,7 @@ describe("OpenAI Chat adapter", () => {
it.effect("maps OpenAI provider options to Chat options", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make().prepare<OpenAIChat.OpenAIChatPayload>(
const prepared = yield* LLMClient.prepare<OpenAIChat.OpenAIChatPayload>(
LLM.request({
model: OpenAI.chat("gpt-4o-mini", { baseURL: "https://api.openai.test/v1/" }),
prompt: "think",
@@ -70,7 +70,7 @@ describe("OpenAI Chat adapter", () => {
)
it.effect("adds native query params to the Chat Completions URL", () =>
LLMClient.make()
LLMClient
.generate(LLM.updateRequest(request, { model: OpenAIChat.model({ ...model, queryParams: { "api-version": "v1" } }) }))
.pipe(
Effect.provide(
@@ -88,7 +88,7 @@ describe("OpenAI Chat adapter", () => {
)
it.effect("uses Azure api-key header for static OpenAI Chat keys", () =>
LLMClient.make()
LLMClient
.generate(
LLM.updateRequest(request, {
model: Azure.model("gpt-4o-mini", {
@@ -116,7 +116,7 @@ describe("OpenAI Chat adapter", () => {
)
it.effect("applies serializable HTTP overlays after payload lowering", () =>
LLMClient.make()
LLMClient
.generate(
LLM.updateRequest(request, {
model: OpenAIChat.model({ ...model, apiKey: "fresh-key", headers: { authorization: "Bearer stale" } }),
@@ -151,7 +151,7 @@ describe("OpenAI Chat adapter", () => {
it.effect("prepares assistant tool-call and tool-result messages", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make().prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_tool_result",
model,
@@ -188,7 +188,7 @@ describe("OpenAI Chat adapter", () => {
it.effect("rejects unsupported user media content", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make()
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_media",
@@ -204,7 +204,7 @@ describe("OpenAI Chat adapter", () => {
it.effect("rejects unsupported assistant reasoning content", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make()
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_reasoning",
@@ -232,7 +232,7 @@ describe("OpenAI Chat adapter", () => {
completion_tokens_details: { reasoning_tokens: 0 },
}),
)
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(body)))
@@ -272,7 +272,7 @@ describe("OpenAI Chat adapter", () => {
deltaChunk({ tool_calls: [{ index: 0, function: { arguments: ':"weather"}' } }] }),
deltaChunk({}, "tool_calls"),
)
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
@@ -298,7 +298,7 @@ describe("OpenAI Chat adapter", () => {
}),
deltaChunk({ tool_calls: [{ index: 0, function: { arguments: ':"weather"}' } }] }),
)
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
@@ -317,7 +317,7 @@ describe("OpenAI Chat adapter", () => {
it.effect("fails on malformed stream chunks", () =>
Effect.gen(function* () {
const body = sseEvents(deltaChunk({ content: 123 }))
const error = yield* LLMClient.make()
const error = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(body)), Effect.flip)
@@ -330,7 +330,7 @@ describe("OpenAI Chat adapter", () => {
const layer = truncatedStream([
`data: ${JSON.stringify(deltaChunk({ role: "assistant", content: "Hello" }))}\n\n`,
])
const error = yield* LLMClient.make()
const error = yield* LLMClient
.generate(request)
.pipe(Effect.provide(layer), Effect.flip)
@@ -340,7 +340,7 @@ describe("OpenAI Chat adapter", () => {
it.effect("fails HTTP provider errors before stream parsing", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make()
const error = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
@@ -360,7 +360,7 @@ describe("OpenAI Chat adapter", () => {
it.effect("short-circuits the upstream stream when the consumer takes a prefix", () =>
Effect.gen(function* () {
const llm = LLMClient.make()
const llm = LLMClient
// The body has more chunks than we'll consume. If `Stream.take(1)` did
// not interrupt the upstream HTTP body the test would hang waiting for
// the rest of the stream to drain.
packages/llm/test/provider/openai-compatible-chat.recorded.test.ts
+4 -4
View File
@@ -55,7 +55,7 @@ const xaiRequest = textRequest({ id: "recorded_xai_text", model: xaiModel })
const xaiToolRequest = weatherToolRequest({ id: "recorded_xai_tool_call", model: xaiModel })
const recorded = recordedTests({ prefix: "openai-compatible-chat", protocol: "openai-compatible-chat" })
const llm = LLMClient.make({ adapters: [OpenAICompatibleChat.adapter, ...OpenRouter.adapters] })
const llm = LLMClient
const openrouterToolLoops = [
{
@@ -128,7 +128,7 @@ describe("OpenAI-compatible Chat recorded", () => {
recorded.effect.with("groq llama 3.3 70b drives a tool loop", { provider: "groq", requires: ["GROQ_API_KEY"], tags: ["tool", "tool-loop", "golden"] }, () =>
Effect.gen(function* () {
expectWeatherToolLoop(yield* runWeatherToolLoop(llm, weatherToolLoopRequest({
expectWeatherToolLoop(yield* runWeatherToolLoop(weatherToolLoopRequest({
id: "recorded_groq_llama_3_3_70b_tool_loop",
model: groqModel,
})))
@@ -158,7 +158,7 @@ describe("OpenAI-compatible Chat recorded", () => {
openrouterToolLoops.forEach((scenario) =>
recorded.effect.with(scenario.name, { provider: "openrouter", requires: ["OPENROUTER_API_KEY"], tags: scenario.tags }, () =>
Effect.gen(function* () {
expectWeatherToolLoop(yield* runWeatherToolLoop(llm, weatherToolLoopRequest({
expectWeatherToolLoop(yield* runWeatherToolLoop(weatherToolLoopRequest({
id: scenario.id,
model: scenario.model,
system: "Use the get_weather tool exactly once, then answer in one short sentence.",
@@ -188,7 +188,7 @@ describe("OpenAI-compatible Chat recorded", () => {
recorded.effect.with("xai grok 4.3 drives a tool loop", { provider: "xai", requires: ["XAI_API_KEY"], tags: ["tool", "tool-loop", "golden", "flagship"] }, () =>
Effect.gen(function* () {
expectWeatherToolLoop(yield* runWeatherToolLoop(llm, weatherToolLoopRequest({
expectWeatherToolLoop(yield* runWeatherToolLoop(weatherToolLoopRequest({
id: "recorded_xai_grok_4_3_tool_loop",
model: xaiFlagshipModel,
})))
packages/llm/test/provider/openai-compatible-chat.test.ts
+4 -4
View File
@@ -54,7 +54,7 @@ const providerFamilies = [
describe("OpenAI-compatible Chat adapter", () => {
it.effect("prepares generic Chat target", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [OpenAICompatibleChat.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.updateRequest(request, {
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
toolChoice: { type: "required" },
@@ -127,7 +127,7 @@ describe("OpenAI-compatible Chat adapter", () => {
it.effect("matches AI SDK compatible basic request body fixture", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [OpenAICompatibleChat.adapter] }).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toEqual({
model: "deepseek-chat",
@@ -145,7 +145,7 @@ describe("OpenAI-compatible Chat adapter", () => {
it.effect("matches AI SDK compatible tool request body fixture", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: [OpenAICompatibleChat.adapter] }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_tool_parity",
model,
@@ -195,7 +195,7 @@ describe("OpenAI-compatible Chat adapter", () => {
it.effect("posts to the configured compatible endpoint and parses text usage", () =>
Effect.gen(function* () {
const response = yield* LLMClient.make({ adapters: [OpenAICompatibleChat.adapter] })
const response = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
packages/llm/test/provider/openai-responses.recorded.test.ts
+2 -2
View File
@@ -41,7 +41,7 @@ const recorded = recordedTests({
protocol: "openai-responses",
requires: ["OPENAI_API_KEY"],
})
const openai = LLMClient.make({ adapters: [OpenAIResponses.adapter] })
const openai = LLMClient
describe("OpenAI Responses recorded", () => {
recorded.effect.with("gpt-5.5 streams text", { tags: ["flagship"] }, () =>
@@ -71,7 +71,7 @@ describe("OpenAI Responses recorded", () => {
recorded.effect.with("gpt-5.5 drives a tool loop", { tags: ["tool", "tool-loop", "golden", "flagship"] }, () =>
Effect.gen(function* () {
expectWeatherToolLoop(yield* runWeatherToolLoop(openai, loopRequest))
expectWeatherToolLoop(yield* runWeatherToolLoop(loopRequest))
}),
)
})
packages/llm/test/provider/openai-responses.test.ts
+14 -14
View File
@@ -29,7 +29,7 @@ const it = testEffect(Layer.empty)
describe("OpenAI Responses adapter", () => {
it.effect("prepares OpenAI Responses target", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make().prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toEqual({
model: "gpt-4.1-mini",
@@ -46,7 +46,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("adds native query params to the Responses URL", () =>
Effect.gen(function* () {
yield* LLMClient.make()
yield* LLMClient
.generate(LLM.updateRequest(request, { model: OpenAIResponses.model({ ...model, queryParams: { "api-version": "v1" } }) }))
.pipe(
Effect.provide(
@@ -66,7 +66,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("uses Azure api-key header for static OpenAI Responses keys", () =>
Effect.gen(function* () {
yield* LLMClient.make()
yield* LLMClient
.generate(
LLM.updateRequest(request, {
model: Azure.model("gpt-4.1-mini", {
@@ -95,7 +95,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("prepares function call and function output input items", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make().prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
id: "req_tool_result",
model,
@@ -121,7 +121,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("maps OpenAI provider options to Responses options", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make().prepare<OpenAIResponses.OpenAIResponsesPayload>(
const prepared = yield* LLMClient.prepare<OpenAIResponses.OpenAIResponsesPayload>(
LLM.request({
model: OpenAI.model("gpt-5.2", { baseURL: "https://api.openai.test/v1/" }),
prompt: "think",
@@ -146,7 +146,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("request OpenAI provider options override model defaults", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make().prepare<OpenAIResponses.OpenAIResponsesPayload>(
const prepared = yield* LLMClient.prepare<OpenAIResponses.OpenAIResponsesPayload>(
LLM.request({
model: OpenAI.model("gpt-4.1-mini", {
baseURL: "https://api.openai.test/v1/",
@@ -179,7 +179,7 @@ describe("OpenAI Responses adapter", () => {
},
},
)
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(body)))
@@ -230,7 +230,7 @@ describe("OpenAI Responses adapter", () => {
},
{ type: "response.completed", response: { usage: { input_tokens: 5, output_tokens: 1 } } },
)
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(
LLM.updateRequest(request, {
tools: [{ name: "lookup", description: "Lookup data", inputSchema: { type: "object" } }],
@@ -264,7 +264,7 @@ describe("OpenAI Responses adapter", () => {
{ type: "response.output_item.done", item },
{ type: "response.completed", response: { usage: { input_tokens: 5, output_tokens: 1 } } },
)
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(body)))
@@ -302,7 +302,7 @@ describe("OpenAI Responses adapter", () => {
{ type: "response.output_item.done", item },
{ type: "response.completed", response: { usage: { input_tokens: 5, output_tokens: 1 } } },
)
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(body)))
@@ -327,7 +327,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("rejects unsupported user media content", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make()
const error = yield* LLMClient
.prepare(
LLM.request({
id: "req_media",
@@ -343,7 +343,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("emits provider-error events for mid-stream provider errors", () =>
Effect.gen(function* () {
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
@@ -357,7 +357,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("falls back to error code when no message is present", () =>
Effect.gen(function* () {
const response = yield* LLMClient.make()
const response = yield* LLMClient
.generate(request)
.pipe(Effect.provide(fixedResponse(sseEvents({ type: "error", code: "internal_error" }))))
@@ -367,7 +367,7 @@ describe("OpenAI Responses adapter", () => {
it.effect("fails HTTP provider errors before stream parsing", () =>
Effect.gen(function* () {
const error = yield* LLMClient.make()
const error = yield* LLMClient
.generate(request)
.pipe(
Effect.provide(
packages/llm/test/provider/openrouter.test.ts
+2 -2
View File
@@ -20,7 +20,7 @@ describe("OpenRouter", () => {
apiKey: "test-key",
})
const prepared = yield* LLMClient.make({ adapters: OpenRouter.adapters }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({ model, prompt: "Say hello." }),
)
@@ -35,7 +35,7 @@ describe("OpenRouter", () => {
it.effect("applies OpenRouter payload options from the model helper", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.make({ adapters: OpenRouter.adapters }).prepare(
const prepared = yield* LLMClient.prepare(
LLM.request({
model: OpenRouter.model("anthropic/claude-3.7-sonnet:thinking", {
providerOptions: {
packages/llm/test/recorded-scenarios.ts
+2 -3
View File
@@ -1,7 +1,6 @@
import { expect } from "bun:test"
import { Effect, Schema, Stream } from "effect"
import { LLM, LLMEvent, type LLMRequest, type LLMResponse, type ModelRef } from "../src"
import type { LLMClient } from "../src/adapter"
import { tool } from "../src/tool"
import { ToolRuntime } from "../src/tool-runtime"
@@ -76,8 +75,8 @@ export const weatherToolLoopRequest = (input: {
: { maxTokens: input.maxTokens ?? 80, temperature: input.temperature ?? 0 },
})
export const runWeatherToolLoop = (client: LLMClient, request: LLMRequest) =>
ToolRuntime.run(client, { request, tools: { [weatherToolName]: weatherRuntimeTool } }).pipe(
export const runWeatherToolLoop = (request: LLMRequest) =>
ToolRuntime.run({ request, tools: { [weatherToolName]: weatherRuntimeTool } }).pipe(
Stream.runCollect,
Effect.map((events) => Array.from(events)),
)
packages/llm/test/tool-runtime.test.ts
+44 -57
View File
@@ -1,7 +1,8 @@
import { describe, expect } from "bun:test"
import { Effect, Layer, Schema, Stream } from "effect"
import { LLM, LLMEvent, LLMRequest } from "../src"
import { LLMClient, RequestExecutor } from "../src/adapter"
import { LLMClient } from "../src/adapter"
import * as AnthropicMessages from "../src/protocols/anthropic-messages"
import * as OpenAIChat from "../src/protocols/openai-chat"
import { tool, ToolFailure } from "../src/tool"
import { ToolRuntime } from "../src/tool-runtime"
@@ -40,11 +41,10 @@ const get_weather = tool({
describe("ToolRuntime", () => {
it.effect("uses the registered model adapter when adding runtime tools", () =>
Effect.gen(function* () {
const llm = LLMClient.make()
const layer = scriptedResponses([sseEvents(deltaChunk({ role: "assistant", content: "Done." }), finishChunk("stop"))])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather } }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather } }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
@@ -70,7 +70,7 @@ describe("ToolRuntime", () => {
}),
)
yield* ToolRuntime.run(LLMClient.make(), {
yield* ToolRuntime.run({
request: LLMRequest.update(baseRequest, {
generation: LLM.generation({ maxTokens: 50 }),
toolChoice: LLM.toolChoice("auto"),
@@ -104,14 +104,13 @@ describe("ToolRuntime", () => {
it.effect("dispatches a tool call, appends results, and resumes streaming", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const layer = scriptedResponses([
sseEvents(toolCallChunk("call_1", "get_weather", '{"city":"Paris"}'), finishChunk("tool_calls")),
sseEvents(deltaChunk({ role: "assistant", content: "It's sunny in Paris." }), finishChunk("stop")),
])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather } }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather } }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
@@ -131,14 +130,13 @@ describe("ToolRuntime", () => {
it.effect("emits tool-error for unknown tools so the model can self-correct", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const layer = scriptedResponses([
sseEvents(toolCallChunk("call_1", "missing_tool", "{}"), finishChunk("tool_calls")),
sseEvents(deltaChunk({ role: "assistant", content: "Sorry." }), finishChunk("stop")),
])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather } }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather } }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
@@ -158,14 +156,13 @@ describe("ToolRuntime", () => {
it.effect("emits tool-error when the LLM input fails the parameters schema", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const layer = scriptedResponses([
sseEvents(toolCallChunk("call_1", "get_weather", '{"city":42}'), finishChunk("tool_calls")),
sseEvents(deltaChunk({ role: "assistant", content: "Done." }), finishChunk("stop")),
])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather } }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather } }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
@@ -179,14 +176,13 @@ describe("ToolRuntime", () => {
it.effect("emits tool-error when the handler returns a ToolFailure", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const layer = scriptedResponses([
sseEvents(toolCallChunk("call_1", "get_weather", '{"city":"FAIL"}'), finishChunk("tool_calls")),
sseEvents(deltaChunk({ role: "assistant", content: "Sorry." }), finishChunk("stop")),
])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather } }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather } }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
@@ -200,11 +196,10 @@ describe("ToolRuntime", () => {
it.effect("stops when the model finishes without requesting more tools", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const layer = scriptedResponses([sseEvents(deltaChunk({ role: "assistant", content: "Done." }), finishChunk("stop"))])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather } }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather } }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
@@ -217,7 +212,6 @@ describe("ToolRuntime", () => {
it.effect("respects maxSteps and stops the loop", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
// Every script entry asks for another tool call. With maxSteps: 2 the
// runtime should run at most two model rounds and then exit even though
// the model still wants to keep going.
@@ -225,7 +219,7 @@ describe("ToolRuntime", () => {
const layer = scriptedResponses([toolCallStep, toolCallStep, toolCallStep])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather }, maxSteps: 2 }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather }, maxSteps: 2 }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
@@ -237,14 +231,13 @@ describe("ToolRuntime", () => {
it.effect("stops when stopWhen returns true after the first step", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const layer = scriptedResponses([
sseEvents(toolCallChunk("call_1", "get_weather", '{"city":"Paris"}'), finishChunk("tool_calls")),
sseEvents(deltaChunk({ role: "assistant", content: "Should not run." }), finishChunk("stop")),
])
const events = Array.from(
yield* ToolRuntime.run(llm, {
yield* ToolRuntime.run({
request: baseRequest,
tools: { get_weather },
stopWhen: (state) => state.step >= 0,
@@ -258,47 +251,42 @@ describe("ToolRuntime", () => {
it.effect("does not dispatch provider-executed tool calls", () =>
Effect.gen(function* () {
// Stub client emits a provider-executed tool-call followed by its
// tool-result and a stop. The runtime must not dispatch a handler (no
// tool-error for unknown name) and must not loop (no second stream).
let streams = 0
const stub: LLMClient = {
prepare: () => Effect.die("not used"),
generate: () => Effect.die("not used"),
stream: () => {
const layer = dynamicResponse((input) =>
Effect.sync(() => {
streams++
return Stream.fromIterable<LLMEvent>([
{ type: "request-start", id: "req_1", model: baseRequest.model },
{
type: "tool-call",
id: "srvtoolu_abc",
name: "web_search",
input: { query: "x" },
providerExecuted: true,
},
{
type: "tool-result",
id: "srvtoolu_abc",
name: "web_search",
result: { type: "json", value: { results: [] } },
providerExecuted: true,
},
{ type: "text-delta", text: "Done." },
{ type: "request-finish", reason: "stop" },
])
},
}
// The runtime's stream type carries `RequestExecutor.Service` because
// adapters use it. Our stub never executes HTTP, but the type still
// demands the service — provide a noop so the test compiles.
const noopExecutor = Layer.succeed(RequestExecutor.Service, {
execute: () => Effect.die("stub client never executes HTTP"),
})
return input.respond(
sseEvents(
{ type: "message_start", message: { usage: { input_tokens: 5 } } },
{ type: "content_block_start", index: 0, content_block: { type: "server_tool_use", id: "srvtoolu_abc", name: "web_search" } },
{ type: "content_block_delta", index: 0, delta: { type: "input_json_delta", partial_json: '{"query":"x"}' } },
{ type: "content_block_stop", index: 0 },
{
type: "content_block_start",
index: 1,
content_block: {
type: "web_search_tool_result",
tool_use_id: "srvtoolu_abc",
content: [{ type: "web_search_result", url: "https://example.com", title: "Example" }],
},
},
{ type: "content_block_stop", index: 1 },
{ type: "content_block_start", index: 2, content_block: { type: "text", text: "" } },
{ type: "content_block_delta", index: 2, delta: { type: "text_delta", text: "Done." } },
{ type: "content_block_stop", index: 2 },
{ type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 8 } },
),
{ headers: { "content-type": "text/event-stream" } },
)
}),
)
const events = Array.from(
yield* ToolRuntime.run(stub, { request: baseRequest, tools: {} }).pipe(
yield* ToolRuntime.run({
request: LLM.updateRequest(baseRequest, { model: AnthropicMessages.model({ id: "claude-sonnet-4-5", apiKey: "test" }) }),
tools: {},
}).pipe(
Stream.runCollect,
Effect.provide(noopExecutor),
Effect.provide(layer),
),
)
@@ -319,7 +307,6 @@ describe("ToolRuntime", () => {
it.effect("dispatches multiple tool calls in one step concurrently", () =>
Effect.gen(function* () {
const llm = LLMClient.make({ adapters: [OpenAIChat.adapter] })
const layer = scriptedResponses([
sseEvents(
deltaChunk({
@@ -335,7 +322,7 @@ describe("ToolRuntime", () => {
])
const events = Array.from(
yield* ToolRuntime.run(llm, { request: baseRequest, tools: { get_weather } }).pipe(
yield* ToolRuntime.run({ request: baseRequest, tools: { get_weather } }).pipe(
Stream.runCollect,
Effect.provide(layer),
),
packages/opencode/src/session/llm-native-tools.ts
+3 -5
View File
@@ -1,6 +1,6 @@
import {
LLM,
type LLMClient,
LLMClient,
type LLMError,
type LLMEvent,
type LLMRequest,
@@ -129,7 +129,6 @@ const dispatchTool = (
// `done` resolves with the accumulated state so the multi-round driver can
// decide whether to recurse.
const runOneRound = (
client: LLMClient,
request: LLMRequest,
tools: Record<string, Tool>,
abort: AbortSignal,
@@ -149,7 +148,7 @@ const runOneRound = (
yield* Effect.forkScoped(
Effect.gen(function* () {
yield* client.stream(request).pipe(
yield* LLMClient.stream(request).pipe(
Stream.runForEach((event) =>
Effect.gen(function* () {
accumulate(state, event)
@@ -219,7 +218,6 @@ const continuationRequest = (request: LLMRequest, state: RoundState): LLMRequest
* interrupted (e.g. via the abort signal).
*/
export const runWithTools = (input: {
readonly client: LLMClient
readonly request: LLMRequest
readonly tools: Record<string, Tool>
readonly abort: AbortSignal
@@ -229,7 +227,7 @@ export const runWithTools = (input: {
const round = (request: LLMRequest, step: number): Stream.Stream<LLMEvent, LLMError, RequestExecutor.Service> =>
Stream.unwrap(
Effect.gen(function* () {
const { events, done } = yield* runOneRound(input.client, request, input.tools, input.abort)
const { events, done } = yield* runOneRound(request, input.tools, input.abort)
const continuation = Stream.unwrap(
Effect.gen(function* () {
const state = yield* Deferred.await(done)
packages/opencode/src/session/llm.ts
+7 -28
View File
@@ -11,14 +11,7 @@ import {
type ProtocolID,
} from "@opencode-ai/llm"
import { RequestExecutor } from "@opencode-ai/llm/adapter"
import {
AnthropicMessages,
BedrockConverse,
Gemini,
OpenAIChat,
OpenAICompatibleChat,
OpenAIResponses,
} from "@opencode-ai/llm/protocols"
import "@opencode-ai/llm/protocols"
import { ProviderTransform } from "@/provider/transform"
import { Config } from "@/config/config"
import { InstanceState } from "@/effect/instance-state"
@@ -499,19 +492,6 @@ const live: Layer.Layer<
// existing AI SDK path. The return shape is deliberately narrow — we are
// not yet committed to native-by-default for any provider.
const NATIVE_PROTOCOLS = new Set<ProtocolID>(["anthropic-messages"])
const NATIVE_ADAPTERS = [
AnthropicMessages.adapter,
OpenAIChat.adapter,
OpenAIResponses.adapter,
Gemini.adapter,
OpenAICompatibleChat.adapter,
BedrockConverse.adapter,
]
const nativeClient = LLMClient.make({
adapters: NATIVE_ADAPTERS,
})
const runNative = Effect.fn("LLM.runNative")(function* (input: StreamRequest, prepared: PreparedStream) {
if (!Flag.OPENCODE_EXPERIMENTAL_LLM_NATIVE) return undefined
if (!input.nativeMessages || input.nativeMessages.length === 0) return undefined
@@ -599,13 +579,12 @@ const live: Layer.Layer<
// subsequent tool-call streaming.
const map = LLMNativeEvents.mapper()
const upstream = filteredNativeTools && filteredNativeTools.length > 0
? LLMNativeTools.runWithTools({
client: nativeClient,
request: llmRequest,
tools: filteredAITools,
abort: input.abort,
})
: nativeClient.stream(llmRequest)
? LLMNativeTools.runWithTools({
request: llmRequest,
tools: filteredAITools,
abort: input.abort,
})
: LLMClient.stream(llmRequest)
return upstream.pipe(
Stream.flatMap((event) => Stream.fromIterable(map.map(event))),
Stream.concat(Stream.unwrap(Effect.sync(() => Stream.fromIterable(map.flush())))),
packages/opencode/test/session/llm-native-stream.test.ts
+4 -27
View File
@@ -1,16 +1,7 @@
import { describe, expect } from "bun:test"
import {
LLMClient,
} from "@opencode-ai/llm"
import { LLMClient } from "@opencode-ai/llm"
import { RequestExecutor } from "@opencode-ai/llm/adapter"
import {
AnthropicMessages,
BedrockConverse,
Gemini,
OpenAIChat,
OpenAICompatibleChat,
OpenAIResponses,
} from "@opencode-ai/llm/protocols"
import "@opencode-ai/llm/protocols"
import { Effect, Layer, Ref, Schema, Stream } from "effect"
import { HttpClient, HttpClientResponse } from "effect/unstable/http"
import { tool, jsonSchema } from "ai"
@@ -100,17 +91,6 @@ const userMessage = (mdl: Provider.Model, id: MessageID, parts: MessageV2.Part[]
parts,
})
// What `runNative` builds. Kept in sync with `session/llm.ts`'s
// NATIVE_ADAPTERS list — if a protocol is added there, add it here.
const adapters = [
AnthropicMessages.adapter,
OpenAIChat.adapter,
OpenAIResponses.adapter,
Gemini.adapter,
OpenAICompatibleChat.adapter,
BedrockConverse.adapter,
]
const it = testEffect(Layer.empty)
describe("LLMNative stream wire-up (audit gap #4 phase 1)", () => {
@@ -128,7 +108,6 @@ describe("LLMNative stream wire-up (audit gap #4 phase 1)", () => {
messages: [userMessage(mdl, userID, [userPart(userID, "Say hello.")])],
})
const client = LLMClient.make({ adapters })
const map = LLMNativeEvents.mapper()
const body = sseBody([
@@ -141,7 +120,7 @@ describe("LLMNative stream wire-up (audit gap #4 phase 1)", () => {
{ type: "message_stop" },
])
const events = yield* client.stream(llmRequest).pipe(
const events = yield* LLMClient.stream(llmRequest).pipe(
Stream.flatMap((event) => Stream.fromIterable(map.map(event))),
Stream.concat(Stream.unwrap(Effect.sync(() => Stream.fromIterable(map.flush())))),
Stream.runCollect,
@@ -246,11 +225,9 @@ describe("LLMNative stream wire-up (audit gap #4 phase 1)", () => {
{ type: "message_stop" },
])
const client = LLMClient.make({ adapters })
const map = LLMNativeEvents.mapper()
const events = yield* LLMNativeTools.runWithTools({
client,
request: llmRequest,
tools: { lookup: aiTool },
abort: new AbortController().signal,
@@ -323,7 +300,7 @@ describe("LLMNative stream wire-up (audit gap #4 phase 1)", () => {
tools: [lookupTool],
})
const prepared = yield* LLMClient.make({ adapters }).prepare(llmRequest)
const prepared = yield* LLMClient.prepare(llmRequest)
expect(prepared.payload).toMatchObject({
tools: [
{
packages/opencode/test/session/llm-native.test.ts
+10 -20
View File
@@ -1,6 +1,6 @@
import { describe, expect } from "bun:test"
import { LLMClient } from "@opencode-ai/llm"
import { AnthropicMessages, BedrockConverse, Gemini, OpenAICompatibleChat, OpenAIResponses } from "@opencode-ai/llm/protocols"
import "@opencode-ai/llm/protocols"
import { Cause, Effect, Exit, Layer, Schema } from "effect"
import { ModelID, ProviderID } from "../../src/provider/schema"
import { LLMNative } from "../../src/session/llm-native"
@@ -598,7 +598,7 @@ describe("LLMNative.request", () => {
tools: [lookupTool],
toolChoice: "lookup",
})
const prepared = yield* LLMClient.make({ adapters: [OpenAIResponses.adapter] }).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toMatchObject({
model: "gpt-5",
@@ -657,7 +657,7 @@ describe("LLMNative.request", () => {
tools: [lookupTool],
toolChoice: "lookup",
})
const prepared = yield* LLMClient.make({ adapters: [AnthropicMessages.adapter] }).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(request.model).toMatchObject({
provider: "anthropic",
@@ -726,7 +726,7 @@ describe("LLMNative.request", () => {
tools: [lookupTool],
toolChoice: "lookup",
})
const prepared = yield* LLMClient.make({ adapters: [OpenAICompatibleChat.adapter] }).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(request.model).toMatchObject({
provider: "togetherai",
@@ -857,7 +857,7 @@ describe("LLMNative.request", () => {
tools: [lookupTool],
toolChoice: "lookup",
})
const prepared = yield* LLMClient.make({ adapters: [Gemini.adapter] }).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(request.model).toMatchObject({
provider: "google",
@@ -929,9 +929,7 @@ describe("LLMNative.request", () => {
system: ["First", "Second", "Third"],
messages: [userMessage(mdl, userID, [textPart(userID, "hello")])],
})
const prepared = yield* LLMClient.make({
adapters: [AnthropicMessages.adapter],
}).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toMatchObject({
system: [
@@ -953,9 +951,7 @@ describe("LLMNative.request", () => {
model: mdl,
messages: messageIds.map((id, index) => userMessage(mdl, id, [textPart(id, `m${index}`)])),
})
const prepared = yield* LLMClient.make({
adapters: [AnthropicMessages.adapter],
}).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toMatchObject({
messages: [
@@ -979,9 +975,7 @@ describe("LLMNative.request", () => {
system: ["You are concise."],
messages: [userMessage(mdl, userID, [textPart(userID, "hello")])],
})
const prepared = yield* LLMClient.make({
adapters: [BedrockConverse.adapter],
}).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toMatchObject({
system: [{ text: "You are concise." }, { cachePoint: { type: "default" } }],
@@ -1006,9 +1000,7 @@ describe("LLMNative.request", () => {
system: ["A", "B", "C"],
messages: ids.map((id, index) => userMessage(mdl, id, [textPart(id, `m${index}`)])),
})
const prepared = yield* LLMClient.make({
adapters: [OpenAIResponses.adapter],
}).prepare(request)
const prepared = yield* LLMClient.prepare(request)
// The serialized OpenAI Responses payload has no cache concept; the
// assertion is that nothing in the payload carries a cache marker.
@@ -1084,9 +1076,7 @@ describe("LLMNative.request", () => {
]),
],
})
const prepared = yield* LLMClient.make({
adapters: [AnthropicMessages.adapter],
}).prepare(request)
const prepared = yield* LLMClient.prepare(request)
expect(prepared.payload).toMatchObject({
messages: [