From e1c6bf92fb99e34fdded46efe3df1860066f250f Mon Sep 17 00:00:00 2001 From: Kit Langton Date: Sun, 26 Apr 2026 12:33:58 -0400 Subject: [PATCH] feat(llm): provider-executed tool pass-through Add a `providerExecuted: boolean` flag to `tool-call` and `tool-result` events plus the persisted `ToolResultPart`. When set, the tool runtime skips client dispatch (the provider already executed the tool) and folds both events into the assistant message so the next round's history carries the call + result for context. Anthropic: decode `server_tool_use` blocks and the three server tool result block types (`web_search_tool_result`, `code_execution_tool_result`, `web_fetch_tool_result`) into `tool-call` / `tool-result` events with `providerExecuted: true`. Round-trip the same parts back into the provider when the assistant message is replayed in subsequent requests. Result block error payloads (`*_tool_result_error`) surface as `result.type === "error"`. OpenAI Responses: decode hosted tool items emitted via `response.output_item.done` (`web_search_call`, `file_search_call`, `code_interpreter_call`, `computer_use_call`, `image_generation_call`, `mcp_call`, `local_shell_call`) as `tool-call` + `tool-result` pairs with `providerExecuted: true`. Each tool's input fields are pulled out explicitly; the full item is passed through as the result payload so consumers can read outputs / sources / status without re-decoding. Tool runtime: extend the dispatch decision so provider-executed tool-calls bypass the handler lookup, and tool-result events with `providerExecuted: true` are appended to the assistant content for round-trip rather than being treated as a separate tool message. Tests: 7 new deterministic fixtures cover Anthropic decode (success + error result + round-trip + unknown server tool name), OpenAI Responses decode (web_search_call, code_interpreter_call), and tool-runtime skip-dispatch. AGENTS.md updates the runtime section to describe pass-through behavior and notes the transport-agnostic design that keeps a future WebSocket adapter (e.g. OpenAI Codex backend) as a sibling rather than a core rewrite. --- packages/llm/AGENTS.md | 13 +- .../llm/src/provider/anthropic-messages.ts | 121 ++++++++++++- packages/llm/src/provider/openai-responses.ts | 70 ++++++++ packages/llm/src/schema.ts | 3 + packages/llm/src/tool-runtime.ts | 26 ++- .../test/provider/anthropic-messages.test.ts | 168 ++++++++++++++++++ .../test/provider/openai-responses.test.ts | 74 ++++++++ packages/llm/test/tool-runtime.test.ts | 64 ++++++- 8 files changed, 530 insertions(+), 9 deletions(-) diff --git a/packages/llm/AGENTS.md b/packages/llm/AGENTS.md index 4ac7ff978e..67c87761cd 100644 --- a/packages/llm/AGENTS.md +++ b/packages/llm/AGENTS.md @@ -47,6 +47,8 @@ Adapters should stay boring and typed: - `toHttp` creates the `HttpClientRequest`. - `parse` decodes provider chunks into `LLMEvent`s. The shared `ProviderShared.sse` helper handles SSE framing, chunk decoding, and stateful chunk-to-event raising; adapters supply `decodeChunk` and a `process` callback that produces events. +The transport is HTTP + SSE today; the `LLMEvent` stream contract is intentionally transport-agnostic. When a provider ships a non-HTTP transport (OpenAI's WebSocket-based Codex backend, hypothetical bidirectional streaming APIs), it should land as a sibling adapter with a `toWs` (or analogous) producer + a `parse` that reads frames from that transport — not by leaking transport details into core types. + ### Patches Patches are the forcing function for provider/model quirks. If a behavior is not universal enough for common IR, keep it as a named patch with a trace entry. Good examples: @@ -133,7 +135,13 @@ Errors must be expressed as `ToolFailure`. The runtime catches it and emits a `t - Input failed the `parameters` Schema. - The handler returned a `ToolFailure`. -Provider-defined tools (e.g. OpenAI built-in `web_search`) should go directly into `request.tools` without a runtime entry. The runtime currently raises `tool-error` for unknown names; if you need pass-through, file an issue. +Provider-defined / hosted tools (e.g. Anthropic `web_search` / `code_execution` / `web_fetch`, OpenAI Responses `web_search_call` / `file_search_call` / `code_interpreter_call` / `mcp_call` / `local_shell_call` / `image_generation_call` / `computer_use_call`) pass through the runtime untouched: + +- Adapters surface the model's call as a `tool-call` event with `providerExecuted: true`, and the provider's result as a matching `tool-result` event with `providerExecuted: true`. +- The runtime detects `providerExecuted` on `tool-call` and **skips client dispatch** — no handler is invoked and no `tool-error` is raised for "unknown tool". The provider already executed it. +- Both events are appended to the assistant message in `assistantContent` so the next round's history carries the call + result for context. Anthropic encodes them back as `server_tool_use` + `web_search_tool_result` (or `code_execution_tool_result` / `web_fetch_tool_result`) blocks; OpenAI Responses callers typically use `previous_response_id` instead of resending hosted-tool items. + +Add provider-defined tools to `request.tools` (no runtime entry needed). The matching adapter must know how to lower the tool definition into the provider-native shape; right now Anthropic accepts `web_search` / `code_execution` / `web_fetch` and OpenAI Responses accepts the hosted tool names listed above. ### Recording Tests @@ -193,7 +201,8 @@ Do not blanket re-record an entire test file when adding one cassette. `RECORD=t - [ ] Build a `Provider.Model` -> `LLM.ModelRef` bridge for OpenCode, including protocol selection, base URLs, headers, limits, capabilities, native provider metadata, and OpenAI-compatible provider family detection. - [ ] Build a `session.llm` -> `LLM.request(...)` bridge for system prompts, message history, tools, tool choice, generation options, reasoning variants, cache hints, and attachments. -- [x] Add a typed `ToolRuntime` that drives the tool loop with Schema-typed parameters/success per tool, single-`ToolFailure` error channel, and `maxSteps`/`stopWhen` controls. Provider-defined tool pass-through is still TODO. +- [x] Add a typed `ToolRuntime` that drives the tool loop with Schema-typed parameters/success per tool, single-`ToolFailure` error channel, and `maxSteps`/`stopWhen` controls. +- [x] Provider-defined tool pass-through: `providerExecuted` flag on `tool-call`/`tool-result` events; Anthropic `server_tool_use` / `web_search_tool_result` / `code_execution_tool_result` / `web_fetch_tool_result` round-trip; OpenAI Responses hosted-tool items decoded as `tool-call` + `tool-result` pairs; runtime skips client dispatch when `providerExecuted: true`. - [ ] Keep auth and deployment concerns in the OpenCode bridge where possible: Bedrock credentials/region/profile, Vertex project/location/token, Azure deployment/API version, and Gateway/OpenRouter routing headers. - [ ] Keep initial OpenCode integration behind a local flag/path until request payload parity and stream event parity are proven against the existing `session/llm.test.ts` cases. diff --git a/packages/llm/src/provider/anthropic-messages.ts b/packages/llm/src/provider/anthropic-messages.ts index 63914c88d9..1d0602b78c 100644 --- a/packages/llm/src/provider/anthropic-messages.ts +++ b/packages/llm/src/provider/anthropic-messages.ts @@ -48,6 +48,35 @@ const AnthropicToolUseBlock = Schema.Struct({ }) type AnthropicToolUseBlock = Schema.Schema.Type +const AnthropicServerToolUseBlock = Schema.Struct({ + type: Schema.Literal("server_tool_use"), + id: Schema.String, + name: Schema.String, + input: Schema.Unknown, + cache_control: Schema.optional(AnthropicCacheControl), +}) +type AnthropicServerToolUseBlock = Schema.Schema.Type + +// Server tool result blocks: web_search_tool_result, code_execution_tool_result, +// and web_fetch_tool_result. The provider executes the tool and inlines the +// structured result into the assistant turn — there is no client tool_result +// round-trip. We round-trip the structured `content` payload as opaque JSON so +// the next request can echo it back when continuing the conversation. +const AnthropicServerToolResultType = Schema.Literals([ + "web_search_tool_result", + "code_execution_tool_result", + "web_fetch_tool_result", +]) +type AnthropicServerToolResultType = Schema.Schema.Type + +const AnthropicServerToolResultBlock = Schema.Struct({ + type: AnthropicServerToolResultType, + tool_use_id: Schema.String, + content: Schema.Unknown, + cache_control: Schema.optional(AnthropicCacheControl), +}) +type AnthropicServerToolResultBlock = Schema.Schema.Type + const AnthropicToolResultBlock = Schema.Struct({ type: Schema.Literal("tool_result"), tool_use_id: Schema.String, @@ -57,7 +86,13 @@ const AnthropicToolResultBlock = Schema.Struct({ }) const AnthropicUserBlock = Schema.Union([AnthropicTextBlock, AnthropicToolResultBlock]) -const AnthropicAssistantBlock = Schema.Union([AnthropicTextBlock, AnthropicThinkingBlock, AnthropicToolUseBlock]) +const AnthropicAssistantBlock = Schema.Union([ + AnthropicTextBlock, + AnthropicThinkingBlock, + AnthropicToolUseBlock, + AnthropicServerToolUseBlock, + AnthropicServerToolResultBlock, +]) type AnthropicAssistantBlock = Schema.Schema.Type type AnthropicToolResultBlock = Schema.Schema.Type @@ -118,6 +153,11 @@ const AnthropicStreamBlock = Schema.Struct({ text: Schema.optional(Schema.String), thinking: Schema.optional(Schema.String), input: Schema.optional(Schema.Unknown), + // *_tool_result blocks arrive whole as content_block_start (no streaming + // delta) with the structured payload in `content` and the originating + // server_tool_use id in `tool_use_id`. + tool_use_id: Schema.optional(Schema.String), + content: Schema.optional(Schema.Unknown), }) const AnthropicStreamDelta = Schema.Struct({ @@ -145,6 +185,7 @@ interface ToolAccumulator { readonly id: string readonly name: string readonly input: string + readonly providerExecuted: boolean } interface ParserState { @@ -200,6 +241,29 @@ const lowerToolCall = (part: ToolCallPart): AnthropicToolUseBlock => ({ input: part.input, }) +const lowerServerToolCall = (part: ToolCallPart): AnthropicServerToolUseBlock => ({ + type: "server_tool_use", + id: part.id, + name: part.name, + input: part.input, +}) + +// Server tool result blocks are typed by name. Anthropic ships three today; +// extend this list when new server tools land. The block content is the +// structured payload returned by the provider, which we round-trip as-is. +const serverToolResultType = (name: string): AnthropicServerToolResultType | undefined => { + if (name === "web_search") return "web_search_tool_result" + if (name === "code_execution") return "code_execution_tool_result" + if (name === "web_fetch") return "web_fetch_tool_result" + return undefined +} + +const lowerServerToolResult = Effect.fn("AnthropicMessages.lowerServerToolResult")(function* (part: ToolResultPart) { + const wireType = serverToolResultType(part.name) + if (!wireType) return yield* invalid(`Anthropic Messages does not know how to round-trip server tool result for ${part.name}`) + return { type: wireType, tool_use_id: part.id, content: part.result.value } satisfies AnthropicServerToolResultBlock +}) + const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (request: LLMRequest) { const messages: AnthropicMessage[] = [] @@ -226,7 +290,11 @@ const lowerMessages = Effect.fn("AnthropicMessages.lowerMessages")(function* (re continue } if (part.type === "tool-call") { - content.push(lowerToolCall(part)) + content.push(part.providerExecuted ? lowerServerToolCall(part) : lowerToolCall(part)) + continue + } + if (part.type === "tool-result" && part.providerExecuted) { + content.push(yield* lowerServerToolResult(part)) continue } return yield* invalid(`Anthropic Messages assistant messages only support text, reasoning, and tool-call content for now`) @@ -337,9 +405,44 @@ const finishToolCall = (tool: ToolAccumulator | undefined) => tool.input || "{}", `Invalid JSON input for Anthropic Messages tool call ${tool.name}`, ) - return [{ type: "tool-call" as const, id: tool.id, name: tool.name, input }] + const event: LLMEvent = tool.providerExecuted + ? { type: "tool-call", id: tool.id, name: tool.name, input, providerExecuted: true } + : { type: "tool-call", id: tool.id, name: tool.name, input } + return [event] }) +// Server tool result blocks come whole in `content_block_start` (no streaming +// delta sequence). We convert the payload to a `tool-result` event with +// `providerExecuted: true`. The runtime appends it to the assistant message +// for round-trip; downstream consumers can inspect `result.value` for the +// structured payload. +const SERVER_TOOL_RESULT_NAMES: Record = { + web_search_tool_result: "web_search", + code_execution_tool_result: "code_execution", + web_fetch_tool_result: "web_fetch", +} + +const isServerToolResultType = (type: string): type is AnthropicServerToolResultType => + type in SERVER_TOOL_RESULT_NAMES + +const serverToolResultEvent = (block: NonNullable): LLMEvent | undefined => { + if (!block.type || !isServerToolResultType(block.type)) return undefined + const errorPayload = + typeof block.content === "object" && block.content !== null && "type" in block.content + ? String((block.content as Record).type) + : "" + const isError = errorPayload.endsWith("_tool_result_error") + return { + type: "tool-result", + id: block.tool_use_id ?? "", + name: SERVER_TOOL_RESULT_NAMES[block.type], + result: isError + ? { type: "error", value: block.content } + : { type: "json", value: block.content }, + providerExecuted: true, + } +} + const processChunk = (state: ParserState, chunk: AnthropicChunk) => Effect.gen(function* () { if (chunk.type === "message_start") { @@ -347,7 +450,11 @@ const processChunk = (state: ParserState, chunk: AnthropicChunk) => return [usage ? { ...state, usage: mergeUsage(state.usage, usage) } : state, []] as const } - if (chunk.type === "content_block_start" && chunk.index !== undefined && chunk.content_block?.type === "tool_use") { + if ( + chunk.type === "content_block_start" && + chunk.index !== undefined && + (chunk.content_block?.type === "tool_use" || chunk.content_block?.type === "server_tool_use") + ) { return [{ ...state, tools: { @@ -356,6 +463,7 @@ const processChunk = (state: ParserState, chunk: AnthropicChunk) => id: chunk.content_block.id ?? String(chunk.index), name: chunk.content_block.name ?? "", input: "", + providerExecuted: chunk.content_block.type === "server_tool_use", }, }, }, []] as const @@ -369,6 +477,11 @@ const processChunk = (state: ParserState, chunk: AnthropicChunk) => return [state, [{ type: "reasoning-delta", text: chunk.content_block.thinking }]] as const } + if (chunk.type === "content_block_start" && chunk.content_block) { + const event = serverToolResultEvent(chunk.content_block) + if (event) return [state, [event]] as const + } + if (chunk.type === "content_block_delta" && chunk.delta?.type === "text_delta" && chunk.delta.text) { return [state, [{ type: "text-delta", text: chunk.delta.text }]] as const } diff --git a/packages/llm/src/provider/openai-responses.ts b/packages/llm/src/provider/openai-responses.ts index 346310af5c..3fe1aa9e2d 100644 --- a/packages/llm/src/provider/openai-responses.ts +++ b/packages/llm/src/provider/openai-responses.ts @@ -94,7 +94,22 @@ const OpenAIResponsesStreamItem = Schema.Struct({ call_id: Schema.optional(Schema.String), name: Schema.optional(Schema.String), arguments: Schema.optional(Schema.String), + // Hosted (provider-executed) tool fields. Each hosted tool item carries its + // own subset of these — we capture them generically so we can surface the + // call's typed input portion and round-trip the full result payload without + // hand-rolling a per-tool schema. + status: Schema.optional(Schema.String), + action: Schema.optional(Schema.Unknown), + queries: Schema.optional(Schema.Unknown), + results: Schema.optional(Schema.Unknown), + code: Schema.optional(Schema.String), + container_id: Schema.optional(Schema.String), + outputs: Schema.optional(Schema.Unknown), + server_label: Schema.optional(Schema.String), + output: Schema.optional(Schema.Unknown), + error: Schema.optional(Schema.Unknown), }) +type OpenAIResponsesStreamItem = Schema.Schema.Type const OpenAIResponsesChunk = Schema.Struct({ type: Schema.String, @@ -275,6 +290,57 @@ const finishToolCall = (tools: Record, item: NonNullabl return [{ type: "tool-call" as const, id: item.call_id, name: item.name, input }] }) +// Hosted tool items (provider-executed) ship their typed input + status + result +// fields all in one item. We expose them as a `tool-call` + `tool-result` pair +// so consumers can treat them uniformly with client tools, only differentiated +// by `providerExecuted: true`. +// +// item.type → tool name. Each entry is the OpenAI Responses item type that +// represents a hosted (provider-executed) tool call. +const HOSTED_TOOL_NAMES: Record = { + web_search_call: "web_search", + web_search_preview_call: "web_search_preview", + file_search_call: "file_search", + code_interpreter_call: "code_interpreter", + computer_use_call: "computer_use", + image_generation_call: "image_generation", + mcp_call: "mcp", + local_shell_call: "local_shell", +} + +const isHostedToolItem = (item: OpenAIResponsesStreamItem): item is OpenAIResponsesStreamItem & { id: string } => + item.type in HOSTED_TOOL_NAMES && typeof item.id === "string" && item.id.length > 0 + +// Pick the input fields the model actually populated when invoking the tool. +// The shape is tool-specific. Keep this list explicit so each tool's input is +// reviewable at a glance — fall back to `{}` for tools we haven't typed yet. +const hostedToolInput = (item: OpenAIResponsesStreamItem): unknown => { + if (item.type === "web_search_call" || item.type === "web_search_preview_call") return item.action ?? {} + if (item.type === "file_search_call") return { queries: item.queries ?? [] } + if (item.type === "code_interpreter_call") return { code: item.code, container_id: item.container_id } + if (item.type === "computer_use_call") return item.action ?? {} + if (item.type === "local_shell_call") return item.action ?? {} + if (item.type === "mcp_call") return { server_label: item.server_label, name: item.name, arguments: item.arguments } + return {} +} + +// Round-trip the full item as the structured result so consumers can extract +// outputs / sources / status without re-decoding. +const hostedToolResult = (item: OpenAIResponsesStreamItem) => { + const isError = typeof item.error !== "undefined" && item.error !== null + return isError + ? ({ type: "error" as const, value: item.error }) + : ({ type: "json" as const, value: item }) +} + +const hostedToolEvents = (item: OpenAIResponsesStreamItem & { id: string }): ReadonlyArray => { + const name = HOSTED_TOOL_NAMES[item.type]! + return [ + { type: "tool-call", id: item.id, name, input: hostedToolInput(item), providerExecuted: true }, + { type: "tool-result", id: item.id, name, result: hostedToolResult(item), providerExecuted: true }, + ] +} + const processChunk = (state: ParserState, chunk: OpenAIResponsesChunk) => Effect.gen(function* () { if (chunk.type === "response.output_text.delta" && chunk.delta) { @@ -306,6 +372,10 @@ const processChunk = (state: ParserState, chunk: OpenAIResponsesChunk) => return [state, events] as const } + if (chunk.type === "response.output_item.done" && chunk.item && isHostedToolItem(chunk.item)) { + return [state, hostedToolEvents(chunk.item)] as const + } + if (chunk.type === "response.completed" || chunk.type === "response.incomplete") { return [state, [{ type: "request-finish" as const, reason: mapFinishReason(chunk), usage: mapUsage(chunk.response?.usage) }]] as const } diff --git a/packages/llm/src/schema.ts b/packages/llm/src/schema.ts index 97a1f04cbb..f916356d14 100644 --- a/packages/llm/src/schema.ts +++ b/packages/llm/src/schema.ts @@ -121,6 +121,7 @@ export const ToolResultPart = Schema.Struct({ id: Schema.String, name: Schema.String, result: ToolResultValue, + providerExecuted: Schema.optional(Schema.Boolean), metadata: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)), }).annotate({ identifier: "LLM.Content.ToolResult" }) export type ToolResultPart = Schema.Schema.Type @@ -262,6 +263,7 @@ export const ToolCall = Schema.Struct({ id: Schema.String, name: Schema.String, input: Schema.Unknown, + providerExecuted: Schema.optional(Schema.Boolean), }).annotate({ identifier: "LLM.Event.ToolCall" }) export type ToolCall = Schema.Schema.Type @@ -270,6 +272,7 @@ export const ToolResult = Schema.Struct({ id: Schema.String, name: Schema.String, result: ToolResultValue, + providerExecuted: Schema.optional(Schema.Boolean), }).annotate({ identifier: "LLM.Event.ToolResult" }) export type ToolResult = Schema.Schema.Type diff --git a/packages/llm/src/tool-runtime.ts b/packages/llm/src/tool-runtime.ts index e3f36bd568..6090a3f1bf 100644 --- a/packages/llm/src/tool-runtime.ts +++ b/packages/llm/src/tool-runtime.ts @@ -10,6 +10,7 @@ import { type LLMEvent, LLMRequest, type ToolCallPart, + type ToolResultPart, type ToolResultValue, } from "./schema" import { ToolFailure } from "./schema" @@ -127,9 +128,30 @@ const accumulate = (state: StepState, event: LLMEvent) => { return } if (event.type === "tool-call") { - const part: ToolCallPart = { type: "tool-call", id: event.id, name: event.name, input: event.input } + const part: ToolCallPart = { + type: "tool-call", + id: event.id, + name: event.name, + input: event.input, + providerExecuted: event.providerExecuted, + } + state.assistantContent.push(part) + // Provider-executed tools are dispatched by the provider; the runtime must + // not invoke a client handler. The matching `tool-result` event arrives + // later in the same stream and is folded into `assistantContent` so the + // next round's message history carries it. + if (!event.providerExecuted) state.toolCalls.push(part) + return + } + if (event.type === "tool-result" && event.providerExecuted) { + const part: ToolResultPart = { + type: "tool-result", + id: event.id, + name: event.name, + result: event.result, + providerExecuted: true, + } state.assistantContent.push(part) - state.toolCalls.push(part) return } if (event.type === "request-finish") { diff --git a/packages/llm/test/provider/anthropic-messages.test.ts b/packages/llm/test/provider/anthropic-messages.test.ts index 82f6b32c55..28a07abcea 100644 --- a/packages/llm/test/provider/anthropic-messages.test.ts +++ b/packages/llm/test/provider/anthropic-messages.test.ts @@ -163,6 +163,174 @@ describe("Anthropic Messages adapter", () => { }), ) + it.effect("decodes server_tool_use + web_search_tool_result as provider-executed events", () => + Effect.gen(function* () { + const body = sseEvents( + { type: "message_start", message: { usage: { input_tokens: 5 } } }, + { type: "content_block_start", index: 0, content_block: { type: "server_tool_use", id: "srvtoolu_abc", name: "web_search" } }, + { type: "content_block_delta", index: 0, delta: { type: "input_json_delta", partial_json: '{"query":"effect 4"}' } }, + { type: "content_block_stop", index: 0 }, + { + type: "content_block_start", + index: 1, + content_block: { + type: "web_search_tool_result", + tool_use_id: "srvtoolu_abc", + content: [{ type: "web_search_result", url: "https://example.com", title: "Example" }], + }, + }, + { type: "content_block_stop", index: 1 }, + { type: "content_block_start", index: 2, content_block: { type: "text", text: "" } }, + { type: "content_block_delta", index: 2, delta: { type: "text_delta", text: "Found it." } }, + { type: "content_block_stop", index: 2 }, + { type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 8 } }, + ) + const response = yield* client({ adapters: [AnthropicMessages.adapter] }) + .generate( + LLM.request({ + ...request, + tools: [{ name: "web_search", description: "Web search", inputSchema: { type: "object" } }], + }), + ) + .pipe(Effect.provide(fixedResponse(body))) + + const toolCall = response.events.find((event) => event.type === "tool-call") + expect(toolCall).toEqual({ + type: "tool-call", + id: "srvtoolu_abc", + name: "web_search", + input: { query: "effect 4" }, + providerExecuted: true, + }) + const toolResult = response.events.find((event) => event.type === "tool-result") + expect(toolResult).toEqual({ + type: "tool-result", + id: "srvtoolu_abc", + name: "web_search", + result: { type: "json", value: [{ type: "web_search_result", url: "https://example.com", title: "Example" }] }, + providerExecuted: true, + }) + expect(LLM.outputText(response)).toBe("Found it.") + expect(response.events.at(-1)).toMatchObject({ type: "request-finish", reason: "stop" }) + }), + ) + + it.effect("decodes web_search_tool_result_error as provider-executed error result", () => + Effect.gen(function* () { + const body = sseEvents( + { type: "message_start", message: { usage: { input_tokens: 5 } } }, + { type: "content_block_start", index: 0, content_block: { type: "server_tool_use", id: "srvtoolu_x", name: "web_search" } }, + { type: "content_block_delta", index: 0, delta: { type: "input_json_delta", partial_json: '{"query":"q"}' } }, + { type: "content_block_stop", index: 0 }, + { + type: "content_block_start", + index: 1, + content_block: { + type: "web_search_tool_result", + tool_use_id: "srvtoolu_x", + content: { type: "web_search_tool_result_error", error_code: "max_uses_exceeded" }, + }, + }, + { type: "content_block_stop", index: 1 }, + { type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } }, + ) + const response = yield* client({ adapters: [AnthropicMessages.adapter] }) + .generate( + LLM.request({ + ...request, + tools: [{ name: "web_search", description: "Web search", inputSchema: { type: "object" } }], + }), + ) + .pipe(Effect.provide(fixedResponse(body))) + + const toolResult = response.events.find((event) => event.type === "tool-result") + expect(toolResult).toMatchObject({ + type: "tool-result", + id: "srvtoolu_x", + name: "web_search", + result: { type: "error" }, + providerExecuted: true, + }) + }), + ) + + it.effect("round-trips provider-executed assistant content into server tool blocks", () => + Effect.gen(function* () { + const prepared = yield* client({ adapters: [AnthropicMessages.adapter] }).prepare( + LLM.request({ + id: "req_round_trip", + model, + messages: [ + LLM.user("Search for something."), + LLM.assistant([ + { + type: "tool-call", + id: "srvtoolu_abc", + name: "web_search", + input: { query: "effect 4" }, + providerExecuted: true, + }, + { + type: "tool-result", + id: "srvtoolu_abc", + name: "web_search", + result: { type: "json", value: [{ url: "https://example.com" }] }, + providerExecuted: true, + }, + { type: "text", text: "Found it." }, + ]), + LLM.user("Thanks."), + ], + }), + ) + + expect(prepared.target).toMatchObject({ + messages: [ + { role: "user", content: [{ type: "text", text: "Search for something." }] }, + { + role: "assistant", + content: [ + { type: "server_tool_use", id: "srvtoolu_abc", name: "web_search", input: { query: "effect 4" } }, + { + type: "web_search_tool_result", + tool_use_id: "srvtoolu_abc", + content: [{ url: "https://example.com" }], + }, + { type: "text", text: "Found it." }, + ], + }, + { role: "user", content: [{ type: "text", text: "Thanks." }] }, + ], + }) + }), + ) + + it.effect("rejects round-trip for unknown server tool names", () => + Effect.gen(function* () { + const error = yield* client({ adapters: [AnthropicMessages.adapter] }) + .prepare( + LLM.request({ + id: "req_unknown_server_tool", + model, + messages: [ + LLM.assistant([ + { + type: "tool-result", + id: "srvtoolu_abc", + name: "future_server_tool", + result: { type: "json", value: {} }, + providerExecuted: true, + }, + ]), + ], + }), + ) + .pipe(Effect.flip) + + expect(error.message).toContain("future_server_tool") + }), + ) + it.effect("rejects unsupported user media content", () => Effect.gen(function* () { const error = yield* client({ adapters: [AnthropicMessages.adapter] }) diff --git a/packages/llm/test/provider/openai-responses.test.ts b/packages/llm/test/provider/openai-responses.test.ts index dbf41c5460..23f44b578c 100644 --- a/packages/llm/test/provider/openai-responses.test.ts +++ b/packages/llm/test/provider/openai-responses.test.ts @@ -158,6 +158,80 @@ describe("OpenAI Responses adapter", () => { }), ) + it.effect("decodes web_search_call as provider-executed tool-call + tool-result", () => + Effect.gen(function* () { + const item = { + type: "web_search_call", + id: "ws_1", + status: "completed", + action: { type: "search", query: "effect 4" }, + } + const body = sseEvents( + { type: "response.output_item.added", item }, + { type: "response.output_item.done", item }, + { type: "response.completed", response: { usage: { input_tokens: 5, output_tokens: 1 } } }, + ) + const response = yield* client({ adapters: [OpenAIResponses.adapter] }) + .generate(request) + .pipe(Effect.provide(fixedResponse(body))) + + const callsAndResults = response.events.filter((event) => event.type === "tool-call" || event.type === "tool-result") + expect(callsAndResults).toEqual([ + { + type: "tool-call", + id: "ws_1", + name: "web_search", + input: { type: "search", query: "effect 4" }, + providerExecuted: true, + }, + { + type: "tool-result", + id: "ws_1", + name: "web_search", + result: { type: "json", value: item }, + providerExecuted: true, + }, + ]) + }), + ) + + it.effect("decodes code_interpreter_call as provider-executed events with code input", () => + Effect.gen(function* () { + const item = { + type: "code_interpreter_call", + id: "ci_1", + status: "completed", + code: "print(1+1)", + container_id: "cnt_xyz", + outputs: [{ type: "logs", logs: "2\n" }], + } + const body = sseEvents( + { type: "response.output_item.done", item }, + { type: "response.completed", response: { usage: { input_tokens: 5, output_tokens: 1 } } }, + ) + const response = yield* client({ adapters: [OpenAIResponses.adapter] }) + .generate(request) + .pipe(Effect.provide(fixedResponse(body))) + + const toolCall = response.events.find((event) => event.type === "tool-call") + expect(toolCall).toEqual({ + type: "tool-call", + id: "ci_1", + name: "code_interpreter", + input: { code: "print(1+1)", container_id: "cnt_xyz" }, + providerExecuted: true, + }) + const toolResult = response.events.find((event) => event.type === "tool-result") + expect(toolResult).toEqual({ + type: "tool-result", + id: "ci_1", + name: "code_interpreter", + result: { type: "json", value: item }, + providerExecuted: true, + }) + }), + ) + it.effect("rejects unsupported user media content", () => Effect.gen(function* () { const error = yield* client({ adapters: [OpenAIResponses.adapter] }) diff --git a/packages/llm/test/tool-runtime.test.ts b/packages/llm/test/tool-runtime.test.ts index 922bc9c3a4..5e7c81f0d7 100644 --- a/packages/llm/test/tool-runtime.test.ts +++ b/packages/llm/test/tool-runtime.test.ts @@ -1,7 +1,8 @@ import { describe, expect } from "bun:test" import { Effect, Layer, Schema, Stream } from "effect" import { LLM, LLMEvent } from "../src" -import { client } from "../src/adapter" +import { client, type LLMClient } from "../src/adapter" +import { RequestExecutor } from "../src/executor" import { OpenAIChat } from "../src/provider/openai-chat" import { tool, ToolFailure } from "../src/tool" import { ToolRuntime } from "../src/tool-runtime" @@ -184,6 +185,67 @@ describe("ToolRuntime", () => { }), ) + it.effect("does not dispatch provider-executed tool calls", () => + Effect.gen(function* () { + // Stub client emits a provider-executed tool-call followed by its + // tool-result and a stop. The runtime must not dispatch a handler (no + // tool-error for unknown name) and must not loop (no second stream). + let streams = 0 + const stub: LLMClient = { + prepare: () => Effect.die("not used"), + generate: () => Effect.die("not used"), + stream: () => { + streams++ + return Stream.fromIterable([ + { type: "request-start", id: "req_1", model: baseRequest.model }, + { + type: "tool-call", + id: "srvtoolu_abc", + name: "web_search", + input: { query: "x" }, + providerExecuted: true, + }, + { + type: "tool-result", + id: "srvtoolu_abc", + name: "web_search", + result: { type: "json", value: { results: [] } }, + providerExecuted: true, + }, + { type: "text-delta", text: "Done." }, + { type: "request-finish", reason: "stop" }, + ]) + }, + } + + // The runtime's stream type carries `RequestExecutor.Service` because + // adapters use it. Our stub never executes HTTP, but the type still + // demands the service — provide a noop so the test compiles. + const noopExecutor = Layer.succeed(RequestExecutor.Service, { + execute: () => Effect.die("stub client never executes HTTP"), + }) + const events = Array.from( + yield* ToolRuntime.run(stub, { request: baseRequest, tools: {} }).pipe( + Stream.runCollect, + Effect.provide(noopExecutor), + ), + ) + + expect(streams).toBe(1) + expect(events.find(LLMEvent.guards["tool-error"])).toBeUndefined() + expect(events.filter(LLMEvent.guards["tool-call"])).toEqual([ + { + type: "tool-call", + id: "srvtoolu_abc", + name: "web_search", + input: { query: "x" }, + providerExecuted: true, + }, + ]) + expect(LLM.outputText({ events })).toBe("Done.") + }), + ) + it.effect("dispatches multiple tool calls in one step concurrently", () => Effect.gen(function* () { const llm = client({ adapters: [OpenAIChat.adapter] })