Wires the prompt-side tool resolver to also surface opencode-native
`Tool.Def[]` alongside the AI SDK record it already builds. With
`OPENCODE_EXPERIMENTAL_LLM_NATIVE=1` set, real production sessions
that satisfy the gate now stream through `LLMNativeTools.runWithTools`
instead of `streamText` — the LLM-native path goes from
"plumbing-only" to "actually used."
Changes:
- `prompt.ts:resolveTools` collects `Tool.Def[]` from the registry
loop and tracks a feasibility flag. MCP tools (which only have AI
SDK shape) flip the flag off; the synthesized `StructuredOutput`
tool that the json_schema branch injects also flips it. The return
shape becomes `{ tools, nativeTools }` where `nativeTools` is
`undefined` whenever any non-registry tool source contributes —
callers fall through to the AI SDK path automatically. The
registry path stays in sync because every `tools[item.id] =
tool({...})` is paired with a `nativeTools.push(item)` at the same
loop iteration.
- The single caller (`prompt.ts:1396`) destructures the new shape
and passes `nativeTools` through to `handle.process(...)`. The
json_schema branch sets `nativeTools = undefined` after injecting
`StructuredOutput` so the gate falls through for structured-output
sessions.
- `runNative` (in `session/llm.ts`) gains two safety nets that work
regardless of caller behavior:
1. Coverage check: if AI SDK tools are non-empty, every key must
have a matching `Tool.Def` in `nativeTools`. A partial set
falls through. Defends against future callers that might
emit a partial native list.
2. Filter parity: `runNative` now calls the existing
`resolveTools(input)` (the in-file permission/user-disabled
filter) and intersects its keys with `nativeTools`, then
feeds the filtered AI SDK record to the dispatcher and the
filtered native list to `LLMNative.request`. Without this,
sessions could see permission-disabled tools advertised on
one path but not the other.
- The dispatch path uses the filtered AI SDK tools record as the
execute table: `LLMNativeTools.runWithTools({ tools:
filteredAITools, ... })`. Tool definitions sent to the model are
the filtered native list. Every tool the model sees can dispatch.
What this enables: a session opted into the experimental flag, with
a clean toolset (registry-only, no MCP, no structured output),
running an Anthropic model, now exercises the streaming-dispatch
loop end-to-end. Tool calls fire as soon as the model finishes
streaming each tool's input; results land in the stream the moment
each handler resolves. Multi-round behavior matches phase 2 step 2b.
What this still does NOT do (deferred to step 4):
- Parity test harness comparing native vs AI SDK event sequences for
the same scripted session. Until that lands, broader confidence
comes from running real sessions with the flag set.
- MCP support on the native path. Sessions with MCP servers
configured stay on AI SDK indefinitely.
- Native support for the synthesized `StructuredOutput` tool.
Verification: opencode typecheck clean for `src/session/*` (the
TUI-side errors visible in the working tree are Kit's parallel
work, untouched here); bridge area tests 36/0/0 across
`llm-native.test.ts` + `llm-native-stream.test.ts` +
`llm-bridge.test.ts`; `prompt.test.ts` still 47/0/0 (no regression
from the resolveTools shape change).
Lands the streaming-dispatch tool loop for the LLM-native path. When
the gate-passing session has `nativeTools` populated, the native
runner forks an AI SDK `tool.execute(...)` the moment a `tool-call`
event arrives mid-stream and injects a synthetic `tool-result` event
back into the same stream when the handler resolves. Long-running
tools no longer block subsequent tool-call streaming; the user sees
each result land as soon as that specific handler completes.
The driver loops across rounds: when a round ends with `reason:
"tool-calls"` AND the dispatchers produced at least one result, the
runner builds a continuation `LLMRequest` (assistant message echoing
text/reasoning/tool-call content + tool messages with results) and
recurses. Stops on a non-`tool-calls` finish, when `maxSteps`
(default 10, mirrors `ToolRuntime.run`) is reached, or when the
underlying scope is interrupted.
New file `session/llm-native-tools.ts`:
- `runWithTools({ client, request, tools, abort, maxSteps? })` is the
public entry point. Returns a `Stream<LLMEvent, LLMError,
RequestExecutor.Service>` of merged model events + synthetic tool
results, ready to flow through `LLMNativeEvents.mapper` for
consumption by the existing session processor.
- `runOneRound` is the internal building block. It opens an unbounded
`Queue<LLMEvent, LLMError | Cause.Done>`, forks a producer that
streams the model and pushes each event to the queue, and forks a
dispatcher (via a scope-bound `FiberSet`) for every
non-provider-executed `tool-call`. Each dispatcher's result is
pushed back into the same queue. After the model stream completes,
the producer awaits `FiberSet.awaitEmpty` and ends the queue;
consumers see end-of-stream. A `Deferred<RoundState>` resolves
alongside so the multi-round driver can decide whether to recurse.
- `dispatchTool` wraps the AI SDK `tool.execute(input, { toolCallId,
messages, abortSignal })` call. Unknown-tool and execute-throws
paths produce `tool-error` events instead of failing the stream
(mirrors `ToolRuntime.run`'s defect-vs-recoverable boundary), so
the model can self-correct on the next round.
Wired into `runNative` (`session/llm.ts`): when `input.nativeTools`
is non-empty, the upstream becomes `LLMNativeTools.runWithTools(...)`
instead of `nativeClient.stream(...)`; the AI SDK `tools` record
flows in as the dispatch table. Zero-tool sessions still take the
direct-stream path (one round, no dispatch overhead).
Mapper update (`session/llm-native-events.ts`): `tool-result` events
whose `result.value` matches the opencode `Tool.ExecuteResult` shape
(`{ output: string, title?: string, metadata?: object }`) now flow
through to the AI-SDK-shaped session event with their `title` and
`metadata` preserved. Provider-executed and synthetic results that
don't match still fall back to `stringifyResult`. Without this, the
session processor would see every native tool result as
`{ title: "", metadata: {}, output: <JSON of the whole record> }`.
Smoke test (`test/session/llm-native-stream.test.ts`): scripts a
two-round Anthropic SSE backend — round 1 issues a `lookup` tool
call, round 2 replies with text after the tool result feeds back.
Asserts the full event sequence threads through `runWithTools`,
the dispatcher, and the mapper:
- `tool-call` event has the streamed JSON input parsed.
- `tool-result` event carries the `ExecuteResult` shape with
`title` + `output` populated (proving the mapper update works).
- Round 2 text-delta arrives after the synthetic tool-result.
- Final `finish` event has `finishReason: "stop"` (loop terminated).
What this still does NOT do (deferred to step 3):
- No production caller populates `nativeTools` yet; that's the
`prompt.ts:resolveTools` change. Until that lands, the gate keeps
every real session on the AI SDK path.
- No parity harness comparing native + AI SDK event sequences for
the same scripted session. That's step 4.
Verification: opencode typecheck clean; 36/0/0 across the three
bridge-area tests; 125/0/0 across the LLM package.
Adds opt-in `nativeTools?: ReadonlyArray<Tool.Def>` to `LLM.StreamInput`
so callers that route through the native path can attach typed
opencode tool definitions alongside the AI SDK `tools` record. The
gate in `runNative` widens accordingly: a session can use the native
path when it has zero tools (existing behavior) OR when it explicitly
provides `nativeTools` matching its AI SDK `tools` (new opt-in). When
`nativeTools` reaches `LLMNative.request`, the existing
`toolDefinition` converter folds each `Tool.Def` into the request's
`tools` array and the LLM core lowers it onto the wire.
This commit deliberately does NOT include the dispatch loop. A
session that opts in by setting `nativeTools` and that triggers a
`tool-call` from the model will see the call event but no
`tool-result` because the native path has no execute handler yet.
That's why no production caller populates `nativeTools`: phase 2
step 2b will land the dispatch loop and only then will real
production sessions route through here.
What this lays in place:
- `StreamInput.nativeTools` typed against `Tool.Def[]` from `@/tool`.
Aliased to `OpenCodeTool` at the import to dodge a clash with the
AI SDK `Tool` type that the same file already imports.
- The `runNative` gate flips from "no tools allowed" to "either no
tools, or `nativeTools` is supplied". An AI SDK tool count > 0
with `nativeTools` undefined still falls through, so existing
production sessions are unaffected.
- `LLMNative.request` already accepted `tools: ReadonlyArray<Tool.Def>`
and converts via `toolDefinition`. We just forward the input
through; no LLM-bridge change.
Smoke coverage: a new test in `llm-native-stream.test.ts` builds a
typed `Tool.Def` (Effect Schema parameters), routes it through
`LLMNative.request` + `LLMClient.prepare`, and asserts the prepared
Anthropic target carries the tool as an `input_schema` block with
the expected JSON Schema shape. This validates the conversion path
that phase 2 step 2b will exercise from inside `runNative`.
Verification: opencode typecheck clean; 35/0/0 across the three
bridge-area tests (`llm-native.test.ts`, `llm-native-stream.test.ts`,
`llm-bridge.test.ts`).
Adds `test/session/llm-native-stream.test.ts` — one focused test that
proves the end-to-end wire-up `runNative` relies on actually produces
session events from a scripted Anthropic SSE response.
The test stays self-contained:
- Builds a fake Anthropic `Provider.Info` + `Provider.Model` via
`ProviderTest`.
- Builds an `LLMRequest` via `LLMNative.request(...)` from a
`MessageV2.WithParts` user message — the same call shape `runNative`
uses inside `session/llm.ts`.
- Creates an `LLMClient` with the same adapters list + `ProviderPatch.defaults`
list as `runNative`. The adapters are imported directly from
`@opencode-ai/llm`; if `runNative`'s `NATIVE_ADAPTERS` array changes,
this test's `adapters` constant has to follow (commented).
- Provides a single fixed-response HTTP layer that returns a scripted
Anthropic SSE body. The layer helper is inlined (12 lines) rather
than imported from `packages/llm/test/lib/http.ts` so the test
doesn't reach across package boundaries.
- Pipes the LLM stream through `LLMNativeEvents.mapper()` exactly as
`runNative` does (`Stream.flatMap` + lazy `Stream.concat` for
flush), runs it to completion, and asserts the key session events:
`text-start` precedes `text-delta`, `finish-step` carries
`finishReason: "stop"`, and `finish` carries the merged usage totals.
This does NOT test the dispatch gate inside `session/llm.ts`
(`!Flag.OPENCODE_EXPERIMENTAL_LLM_NATIVE`, missing `nativeMessages`,
tools present, non-Anthropic protocol). Those are simple boolean
conditions and don't need separate coverage. It also does not exercise
the production `Service` layer — that's deferred to Phase 2 step 2
(tool support) and Phase 2 step 3 (production caller wiring).
What the test buys: confidence that the conversion pipeline works and
catches regressions in `LLMNative.request`, the LLM adapter set, or
`LLMNativeEvents.mapper` before they would surface in a real session.
Verification: 34/0/0 across the three bridge-area tests
(`llm-native.test.ts` + `llm-native-stream.test.ts` +
`llm-bridge.test.ts`); opencode typecheck clean.
Adds the parallel `runNative()` path inside `session/llm.ts` so a narrow
slice of sessions can flow through `@opencode-ai/llm` instead of the AI
SDK `streamText`. Behavior is gated and shipped off by default; only
callers that opt in see any difference.
The full migration plan (audit gap #4) is parallel-path-with-flag,
prove parity test-by-test, flip default last. This commit is phase 1:
get the wire-up in place behind a flag with one protocol so we can see
whether the design holds before committing to the full migration.
Wire-up summary:
- New flag `OPENCODE_EXPERIMENTAL_LLM_NATIVE` (also enabled by the
umbrella `OPENCODE_EXPERIMENTAL`). Off by default.
- The session-LLM `live` layer now consumes `RequestExecutor.Service`,
and the `defaultLayer` provides `RequestExecutor.defaultLayer` so a
Node fetch HTTP client backs every native stream.
- `runNative(input)` returns `Stream<Event> | undefined`. `undefined`
means "fall through to AI SDK." It returns a real stream only when
every gate passes: the flag is set, the caller populated
`input.nativeMessages` (the bridge needs typed `MessageV2.WithParts`,
not the AI SDK `messages` array), the session has zero tools (Phase
2 will lift this), and the bridge routes the model to a protocol in
`NATIVE_PROTOCOLS`.
- `NATIVE_PROTOCOLS` is a single-entry set today: `anthropic-messages`.
Other adapters are imported and registered with the client so the
Phase 2 expansion is a one-line edit, not an architecture change.
- Stream wiring: client.stream(req) -> Stream.flatMap(event ->
fromIterable(map.map(event))) -> Stream.concat(suspended
fromIterable(map.flush())) -> Stream.provideService(
RequestExecutor.Service, executor). The flush stream is built lazily
with `Stream.unwrap(Effect.sync(...))` so it observes the mapper
final state after every upstream event has been mapped.
- The mapper (`LLMNativeEvents.mapper`) emits AI-SDK-shaped session
events from `LLMEvent` so downstream consumers see one shape.
What this does NOT do (deferred to later phases):
- No tool support on the native path (skipped, falls through).
- No parity harness yet; Phase 2 builds it.
- No production traffic; flag is off by default and no production
caller populates `nativeMessages`.
- No reasoning/cache/multi-modal coverage. Anthropic supports reasoning
and cache via existing patches, so those start working as soon as a
caller routes a real session through.
Verification: opencode typecheck clean, bridge tests still green
(33/0/0 across llm-native.test.ts + llm-bridge.test.ts); LLM package
tests green (123/0/0).
Five review findings; all small, all independent.
H2: Bedrock used raw `JSON.parse` and `JSON.stringify` despite the
package rule against ad-hoc JSON encoders. The in-loop parse on each
event-stream frame goes through `ProviderShared.parseJson` (yielded
inside `Effect.gen`); the `decodeChunk` error fallback uses
`ProviderShared.encodeJson` instead of `JSON.stringify` for the raw
field on `ProviderChunkError`. No behavior change — just channels
JSON through the shared Schema-driven codec.
H3: `BedrockConverse.toHttp` built a `baseHeaders` record with
`content-type: application/json` and passed it through both auth
paths. The bearer path called `jsonPost` with the raw model headers
(no manual content-type), the SigV4 path used `baseHeaders` plus the
signed result. Two paths produced subtly different header sets and
both relied on `jsonPost` overwriting/adding the same content-type
key. Simplify: drop the unused bearer-side construction; rename the
SigV4 input to `headersForSigning` and document why content-type
must be present at signing time (signature covers it).
M4: Lift `isRecord` from `gemini.ts` into `ProviderShared.isRecord`
so adapters share one definition. The duplicates in `llm.ts` (LLM IR
layer) and `llm-native.ts` (OpenCode bridge) stay where they are —
those are at different layers and importing from `provider/` would
invert the dependency direction. Net effect: the provider layer
goes from 2 copies to 1.
L8: `TransportError` lost everything but the message string.
Surface the originating reason tag (`Timeout` / `TransportError` /
`ResponseError` / `RequestError`) and the request URL when
available, both as optional Schema fields. Consumers that don't
care keep getting the same `message` rendering; consumers that do
can finally render "timed out connecting to https://..." instead
of "HTTP transport failed".
M9 + L3: Two dead branches. Anthropic's `processChunk` had
`?? ""` fallbacks for `partial_json` after an early-return guard
already proved it non-empty. OpenAI Chat's `mapFinishReason` had
`if (reason === undefined || reason === null) return "unknown"`
followed by `return "unknown"` — both branches went to the same
place. Drop the unreachable code.
120 LLM-package tests + 33 OpenCode bridge tests still green.
Three review findings collapsed into one ProviderShared pass.
M1: Five adapters duplicated the same six-line block:
const ChunkJson = Schema.fromJsonString(Chunk)
const TargetJson = Schema.fromJsonString(Target)
const decodeChunkSync = Schema.decodeUnknownSync(ChunkJson)
const encodeTarget = Schema.encodeSync(TargetJson)
const decodeTarget = Schema.decodeUnknownEffect(Draft.pipe(Schema.decodeTo(Target)))
const decodeChunk = (data) => Effect.try({...chunkError(...)})
Lift it into `ProviderShared.codecs({ adapter, draft, target, chunk,
chunkErrorMessage })` returning `{ encodeTarget, decodeTarget,
decodeChunk }`. The result drops directly into `Adapter.define`'s
`validate` field (uses `validateWith` internally to map parse errors
to InvalidRequestError). Adopted in OpenAI Chat, OpenAI Responses,
Anthropic Messages, and Gemini. Bedrock has a custom event-stream
`decodeChunk` that takes `unknown` (not `string`) so it keeps its
inline codecs.
M2: Four adapters defined an identical `ToolAccumulator` interface
(`{ readonly id: string; readonly name: string; readonly input:
string }`). Lift to `ProviderShared.ToolAccumulator`. Anthropic
extends it locally with `providerExecuted` for hosted tools.
M3: The five `mapUsage` implementations had subtly different
`totalTokens` policies — OpenAI Chat passed through whatever the
provider sent, OpenAI Responses unconditionally summed inputs and
output (publishing `totalTokens: 0` when both were `undefined`),
Anthropic and Gemini guarded with conditionals, Bedrock used a
`(...) || undefined` falsy fallback. Add `ProviderShared.totalTokens`
with one rule: prefer provider-supplied total, else sum inputs and
outputs only when at least one is defined, else `undefined`. Fixes
the OpenAI Responses `totalTokens: 0` bug.
M6: Anthropic's `mergeUsage` recomputed `totalTokens` from the merged
input/output via two nested ?? chains and a conditional sum.
Simplified to use the same totalTokens helper, with `inputTokens` and
`outputTokens` extracted as locals so the merge is one ?? per field
and the comment explains why merging exists (Anthropic emits usage
on `message_start` and `message_delta`).
No behavior changes other than the OpenAI Responses fix; existing
tests pass unchanged. 120 LLM-package tests + 33 OpenCode bridge
tests green.
Two issues from the review of the LLM package's six adapters.
H1: Inconsistent apiKey precedence. Five of six adapters spread the
caller's headers first then set the auth header (apiKey wins), but
`OpenAICompatibleChat.model` did the opposite (caller headers won).
That meant a user passing both `apiKey` and `headers.authorization`
would get auth from a different source depending on which adapter
they routed through. Flip the OpenAI-compatible adapter to match the
rest, and add a comment documenting the rule: apiKey wins, callers
who want their own auth header should omit `apiKey` entirely.
H4: Gemini tool-schema sanitization was split across two functions
that both ran on every Gemini request — `convertJsonSchema` in the
adapter (lossy projection: drop empty objects, derive nullable from
type-array, allowlist of preserved keys, recursive properties/items)
and `sanitizeGeminiSchemaNode` registered as a default `tool-schema`
patch (fix-up: integer enums to strings, dangling required filtering,
untyped array typing, scalar property stripping). Both passes only
ran on Gemini models; debugging a tool schema rejection meant
checking both files.
Fold the patch's rules into the adapter as `sanitizeToolSchemaNode`,
running before the existing projection step (renamed
`projectToolSchemaNode`). Compose them in `convertToolSchema` and use
that in `lowerTool`. Delete the patch from `provider/patch.ts` and
`ProviderPatch.defaults`. The behavior is unchanged — same input,
same output — but the rules now live in one file with a header
comment explaining the two concerns.
The matching test in `gemini.test.ts` no longer needs to opt into a
patch list; it now asserts the adapter alone produces the sanitized
shape.
Closes audit gap #3. The bridge now extracts the encrypted reasoning
blob from `MessageV2.ReasoningPart.metadata` and surfaces it on
`LLM.ReasoningPart.encrypted`, where the Anthropic and Bedrock
adapters lower it to the wire — Anthropic emits `thinking.signature`,
Bedrock emits `reasoningContent.reasoningText.signature`. Without
this, multi-turn sessions with reasoning models would lose the
encrypted state on every step and break the chain.
The encrypted blob originates in three different places depending on
how the session was started:
1. AI-SDK Anthropic sessions store it as
`metadata.anthropic.signature` (per AI SDK provider-keyed
convention).
2. AI-SDK OpenAI sessions store it as
`metadata.openai.reasoningEncryptedContent`.
3. Future LLM-native sessions will store it as a top-level
`metadata.encrypted` string (cleanest shape — provider-agnostic,
matches the LLM IR field name).
The new `encryptedReasoning` helper probes all three locations in
order, so existing OpenCode sessions can be served by the LLM-native
path without re-recording reasoning content. The full `metadata`
record continues to flow through to `LLM.ReasoningPart.metadata`
unchanged, preserving any provider-specific fields adapters might
read in the future.
OpenAI Responses encrypted reasoning round-trip is intentionally out
of scope: the LLM-package adapter doesn't yet model reasoning items
in the request body. That's a separate adapter feature requiring new
input-item schema variants and is deferred until needed.
Tests (5 new in llm-native.test.ts):
- AI-SDK Anthropic signature extracted into LLM.ReasoningPart.encrypted.
- End-to-end Anthropic lowering: bridge \u2192 client.prepare \u2192 target with
`thinking.signature` populated correctly.
- AI-SDK OpenAI reasoningEncryptedContent extracted (forward
compatibility — useful when the OpenAI Responses adapter gains
reasoning-item lowering).
- Top-level metadata.encrypted extracted (LLM-native session shape).
- No known key in metadata leaves `encrypted` undefined.
Verified: 33/0/0 across native + bridge tests (was 28; +5 from the
new reasoning extraction tests).
Closes audit gap #2 (FilePart \u2192 MediaPart not implemented).
The bridge now lowers `MessageV2.FilePart` on user messages into
`LLM.MediaPart`, unblocking image and document inputs. The first
pass supports `data:` URLs only — the inline base64 form most
commonly produced by the OpenCode UI for pasted screenshots and
attached files. `http(s):` and `file:` URLs are explicitly
rejected with a clear error so a future fetch / filesystem-read
path can plug in cleanly without regressing safety.
Implementation:
- New `lowerFilePart` helper extracts the base64 payload from a
data URL via a single regex; failure yields a typed
`UnsupportedContentError` carrying both the partType and a
`reason` that includes the offending URL for debuggability.
- New `lowerUserPart` dispatches user-side parts: text \u2192
`LLM.text`, file \u2192 `MediaPart`. Returns identity-empty
for any unsupported part type the static gate would have caught.
- `userMessage` is now `Effect.fnUntraced` so file conversion can
yield typed errors. `lowerMessage` (the per-message dispatcher,
renamed from `messages` to free the local name) cascades the
Effect through the request flow via `Effect.forEach`.
- `supportsPart` static gate now allows `file` parts on user
messages. Assistant messages still reject file parts (the LLM
IR's MediaPart isn't valid in assistant content for any
adapter we ship today).
- `UnsupportedContentError` gains an optional `reason` field that
appends to the canonical message as `<base>: <reason>`. Existing
static-gate failures keep the same shape (no reason).
Tests (3 new, 1 rewritten):
- Image data URL with filename round-trips to MediaPart with
base64-stripped data.
- PDF data URL preserves filename and base64 payload.
- `https:` URL rejected with an error mentioning both the file
partType, the message ID, and the offending URL.
- The pre-existing "fails instead of dropping unsupported native
parts" test now uses a reasoning part on a user message
(reasoning is valid for assistants only) since file parts with
data URLs are no longer rejected by the static gate.
Out of scope, intentional follow-ups:
- HTTP/HTTPS URL fetching (would need HttpClient.HttpClient and a
decision on caching, retries, size limits).
- File path / file:// URL reading (would need FileSystem.FileSystem
and a permission check against the session's working directory).
- File parts on assistant messages (LLM IR doesn't model
assistant-side media; defer until we hit a provider that needs it).
- text/plain and application/x-directory file parts that the
AI-SDK path converts to text inline at message-v2.ts:791 — for
the bridge, those should be converted upstream before reaching
LLMNative.request rather than handled here.
Verified: bun typecheck clean, 28/0/0 across native + bridge
tests (was 21; +7 from the FilePart additions plus the rewritten
unsupported-parts test).
Lift the prompt-cache policy out of OpenCode's bridge and into the
LLM package as a typed, gated patch. The policy mirrors the AI-SDK
applyCaching path (packages/opencode/src/provider/transform.ts:229):
mark the first 2 system parts and the last 2 messages with an
ephemeral cache hint, gated on `model.capabilities.cache.prompt`.
Adapters lower the hint structurally — Anthropic emits
`cache_control: { type: "ephemeral" }` on the marked block,
Bedrock emits a positional `cachePoint: { type: "default" }`
after the marked block (added in 9d7d518ac). The capability gate
keeps non-cache adapters (OpenAI Responses, Gemini, OpenAI-compat
Chat) hint-free.
Why a Patch and not bridge code:
- packages/llm/AGENTS.md TODO explicitly calls for cache hint patches
- Other consumers of @opencode-ai/llm get caching for free
- The bridge stays focused on shape conversion (MessageV2 \u2192 LLMRequest)
- Patches compose via ProviderPatch.defaults (now includes this one)
- The capability gate is a typed predicate, not provider-name matching
Implementation:
- New `cachePromptHints` patch in provider/patch.ts. The
`withCacheOnLastText` helper uses Array.findLastIndex (codebase
idiom) and short-circuits when no text part exists so messages
with only tool-result content are returned identity-equal.
- `EPHEMERAL_CACHE` is a single shared CacheHint instance — no
per-request allocation, preserves `instanceof` for any consumer
that checks class identity.
- Added to `ProviderPatch.defaults` so existing callers that pass
`defaults` get cache support automatically.
Tests (5 new in patch.test.ts):
- Marks first 2 system parts on cache-capable models.
- Marks last text part of last 2 messages.
- Targets the last text part when a message has trailing
non-text content (assistant text + tool-call).
- Returns content unchanged (identity-equal) when no text part
exists, so pure tool-result messages don't allocate.
- No-op when the model does not advertise prompt caching.
Bridge cleanup:
- Removed `applyCachePolicy`, `withCacheOnLastText`,
`updateMessageContent`, `EPHEMERAL_CACHE` from llm-native.ts
(-30 lines of bridge-side cache code).
- Dropped now-unused `CacheHint`, `LLMRequest`, `Message` imports.
- The bridge's only responsibility is now MessageV2 lowering;
callers wire `patches: ProviderPatch.defaults` at client
construction.
OpenCode tests rewritten:
- Old: assert on `request.system[N].cache` (bridge internals).
- New: assert on `prepared.target` after running through
`LLMClient.make({ adapters, patches: ProviderPatch.defaults })
.prepare(request)` — verifies the full lowering end-to-end.
- Anthropic: target.system[0..1] carry `cache_control: ephemeral`,
target.messages[1..2] carry it on the final text block.
- Bedrock: target has `cachePoint` markers after each cached block.
- Non-cache (OpenAI Responses): JSON.stringify(target) contains
none of `cache_control` / `cachePoint` / `ephemeral`.
Verified: bun typecheck clean across both packages, 120/0/0 in LLM
package (was 113; +7 from new patch tests counting parameter
variations), 21/0/0 in OpenCode native+bridge tests.
Close the parity gaps deferred from the original Bedrock pass.
Schema additions on the Converse target:
- BedrockImageBlock for { image: { format, source: { bytes } } }.
Supported formats per Converse docs: png, jpeg, gif, webp.
- BedrockDocumentBlock for { document: { format, name, source: { bytes } } }.
Supported formats: pdf, csv, doc, docx, xls, xlsx, html, txt, md.
- BedrockCachePointBlock for the positional { cachePoint: { type } }
marker. Currently emits the only Bedrock cache type, 'default'. A
TODO marks where to map ttlSeconds → ttl ('5m' | '1h') once we have
a recorded cassette to validate the wire shape.
Lowering:
- TextPart and SystemPart cache hints emit a positional cachePoint
marker right after their text block. Both 'ephemeral' and
'persistent' CacheHint types map onto Bedrock's 'default' since
Bedrock does not distinguish — this matches the convention the
Anthropic adapter uses (cache?.type === 'ephemeral' check).
- MediaPart routes by mediaType: 'image/*' → image block, everything
else → document block. MIME type → format mapping is via
IMAGE_FORMATS / DOCUMENT_FORMATS records typed with 'as const
satisfies' so the keys stay narrow at compile time.
- A small textWithCache helper collapses the 'push text, push
cachePoint if cache is set' pattern that would otherwise repeat at
three callsites (system, user-text, assistant-text).
- Bytes are encoded via ProviderShared.mediaBytes — the shared
helper Kit landed in c3346f7dc.
Bug fix: lowerSystem was dead code in the previous draft. The
prepare() function still inlined the pre-cache .map(...) that
discarded system cache hints. prepare() now calls lowerSystem so
the cachePoint markers actually flow through.
Tests (7 new fixtures, all green):
- Cache hint on system / user-text / assistant-text emits cachePoint
after text in each context.
- No cache hint → no cachePoint emitted (regression guard).
- Image lowering covers png / jpeg / jpg-alias / webp.
- Uint8Array image bytes are base64-encoded ([1,2,3,4,5] → AQIDBAU=).
- Document lowering with filename round-trip and missing-filename
fallback to 'document.<format>'.
- Unsupported image MIME (image/svg+xml) is rejected with a clear
error message.
- Unsupported document MIME (application/x-tar) is rejected with a
clear error message.
Recorded cassettes for cache hints, images, and documents are still
TODO — the wire shapes are exercised deterministically here and will
be validated against a live model in a follow-up cassette pass.
Verified: bun typecheck clean, 113 pass / 0 fail / 0 skip (was 106;
+7 from the new fixture tests).
Phase A continuation of the ProviderShared dedupe pass. Three more
patterns lifted into ProviderShared so they're written once:
ProviderShared.invalidRequest(message) — replaces six identical
`const invalid = (message) => new InvalidRequestError({ message })`
one-liners across openai-chat, openai-responses, anthropic-messages,
gemini, openai-compatible-chat, and bedrock-converse. Each adapter
keeps a short `const invalid = ProviderShared.invalidRequest` alias
so the 27 callsite `yield* invalid("...")` patterns are unchanged.
Bedrock's SigV4 catch path and the openai-compatible-chat baseURL
guard both go through the helper now too.
ProviderShared.validateWith(decode) — replaces the identical
`(draft) => decode(draft).pipe(Effect.mapError((e) =>
invalid(e.message)))` lambda body in five adapters. Same line count
but shorter, names the pattern, and keeps the `decode → mapError →
InvalidRequestError` translation in one canonical spot.
ProviderShared.jsonPost({ url, body, headers }) — replaces the
five-adapter pattern of `HttpClientRequest.post(url).pipe(setHeaders,
bodyText)` for JSON-body POSTs. Sets `content-type: application/json`
last so caller headers can override everything except the
content-type. Bedrock uses it for both the bearer-auth and SigV4-
signed paths; SigV4 still signs against `baseHeaders` (which already
contained content-type) so the signature matches what the helper
ultimately sends.
Net change: -73 / +86 (+13 in shared.ts mostly JSDoc; -86 across the
six adapters). The `HttpClientRequest` and `InvalidRequestError`
imports are dropped from the five SSE adapters and from Bedrock since
they're no longer referenced directly.
Verified: `bun typecheck` clean, 106 pass / 0 fail / 0 skip
(unchanged).
Update the adapter authoring guide to reflect the dedupe pass:
- Generalize the `parse` bullet from `ProviderShared.sse` to
`ProviderShared.framed` and call out the two framing dialects
in use today (SSE for OpenAI/Anthropic/Gemini/compat, AWS event
stream for Bedrock).
- Spell out that `framed`'s `framing` parameter is the seam for
new wire formats; the rest of the pipeline is shared.
- New 'Shared adapter helpers' subsection enumerating the
`ProviderShared` exports a new adapter author should reach for
before hand-rolling: `framed`, `sse`, `sseFraming`, `joinText`,
`parseToolInput`, `parseJson`, `chunkError`.
- Closing nudge: lift 3-5 line repeats into ProviderShared rather
than copy them between adapters.
Doc-only — no code or test changes.
Promote three repeated patterns out of individual adapters into
ProviderShared so a fifth or sixth adapter doesn't write the same
glue code over again.
ProviderShared.joinText(parts) — replaces the per-adapter `text()`
helper that joined an array of parts with newlines. Used by OpenAI
Chat (system content, user text, assistant text), OpenAI Responses
(system content), and Gemini (systemInstruction). The dead copies in
Anthropic Messages and Bedrock are gone.
ProviderShared.parseToolInput(adapter, name, raw) — replaces the
identical `parseJson(adapter, raw || "{}", \`Invalid JSON input
for <adapter> tool call <name>\`)` invocation in finishToolCall
across Anthropic, OpenAI Chat, OpenAI Responses, and Bedrock. Uniform
error message and the empty-string-to-"{}" fallback handled in one
place.
ProviderShared.framed(...) — generalizes the existing `sse()` helper
so the protocol-specific framing layer is pluggable. The shared
shape is bytes → frames → chunk → (state, events) with mapError /
mapEffect / mapAccumEffect / catchCause as the spine; framing is
the only varying step.
ProviderShared.sseFraming — the SSE-specific framing implementation
(decodeText + Sse.decode + filter [DONE]). The existing `sse()`
helper now delegates to `framed` with this framing, keeping the
adapter API surface identical.
Bedrock's parseStream — collapses to a single `ProviderShared.framed`
call with its own `eventStreamFraming` step. The cursor-based byte
buffer + AWS event-stream codec live as inputs to framed; everything
else is shared with the SSE adapters. Bedrock now has the same
`catchCause → streamError` terminal-error normalization that SSE
adapters have (it was missing before this refactor).
Net effect across the llm package: -66 lines / +114 lines but the
+114 is mostly JSDoc on the new helpers; adapter implementations
shrink. A future protocol (Bedrock InvokeModel, Vertex Gemini binary
streaming, etc.) plugs in by supplying its `framing` step.
Verified: `bun typecheck` clean, 106 pass / 0 fail / 0 skip
(unchanged from before the refactor).
Cleanup of the Bedrock adapter (ba1705d) following parallel review
passes for code reuse, code quality, and efficiency.
- Drop dead `text` join helper and unused `TextPart` import.
- Schema-validate `model.native.aws_credentials` instead of seven
manual `typeof` guards in `credentialsFromInput`. Removes the
unsafe `as Record<string, unknown>` cast and fixes the dead
`native?.region` fallback (the `model()` constructor only writes
`aws_region`).
- Skip the JSON.parse → JSON.stringify → Schema.fromJsonString triple
round-trip in the frame consumer. The eventstream codec already
hands us a UTF-8 payload; parse once and feed the wrapped object
directly to `Schema.decodeUnknownSync(BedrockChunk)`.
- Replace O(n²) buffer concat in `consumeFrames` with a cursor-based
state `{ buffer, offset }`. Compaction happens once per network
chunk via `appendChunk` instead of per frame; frame slicing is
zero-copy via `subarray`. Bounded buffer growth regardless of
stream length.
- Rename `ParserState.finishReason` → `pendingStopReason` (raw
string) and defer the `mapFinishReason` call to the single emit
site, plus the `onHalt` fallback. Tightens the helper's signature
to `(reason: string)` so the chunk-typed `messageStop.stopReason`
flows through without the optional widening.
- Restructure `signRequest` to take an object parameter (was four
positional args), and replace the manual `forEach`-into-record with
`Object.fromEntries(signed.headers.entries())`.
- Inline single-use `status` and `useTools` variables.
- Widen `fixedResponse` to accept `ConstructorParameters<Response>[0]`
so binary fixtures (`Uint8Array`, streams) flow without casts. The
Bedrock test's `fixedBytes` helper now wraps it cleanly.
- Tidy `captureResponseBody` into a ternary returning the union shape
directly so the call site spreads the captured object without
reaching for `bodyEncoding` explicitly.
Verified: `bun typecheck` clean, 106 pass / 0 fail / 0 skip
(unchanged from before the refactor).
Implements the AWS Bedrock Converse streaming protocol as the 5th
first-class adapter in @opencode-ai/llm. Single `bedrock-converse`
adapter covers all underlying models (Anthropic, Llama, Mistral,
Cohere, Nova, Titan) since Converse is uniform.
Wire format: messages with text / reasoning / toolUse / toolResult
content blocks, system blocks, inferenceConfig, toolConfig with
toolSpec + toolChoice. Image / document / cache-point content types
are still TODO.
Streaming: AWS event stream binary framing via @smithy/eventstream-codec.
Each frame is decoded then dispatched by `:event-type` header into
the chunk schema. Bedrock splits the finish across `messageStop`
(reason) and `metadata` (usage) — the parser stashes the reason and
emits a single consolidated `request-finish` event when metadata
arrives, with an `onHalt` fallback for truncated streams.
Auth: two paths. Bearer API key (newer) when the consumer sets
`model.headers.authorization = 'Bearer <key>'`. SigV4 signing via
aws4fetch otherwise — credentials live on `model.native.aws_credentials`
and are signed at `toHttp` time so STS-vended tokens are picked up
when the consumer rebuilds the model. The adapter rejects requests
with neither auth path with a clear InvalidRequestError.
Routing: `@ai-sdk/amazon-bedrock` lowers to `bedrock-converse` via
the new `AmazonBedrock` provider routing module; the OpenCode
`llm-bridge.ts` registers it.
Cassette format: response bodies under
`application/vnd.amazon.eventstream` and `application/octet-stream`
content types are now stored as base64 with `bodyEncoding: 'base64'`
on the response snapshot — text round-tripping mangled the CRC32
fields in event-stream frames. Existing cassettes (SSE/JSON) omit
the field and decode as text unchanged.
Tests: 11 deterministic fixtures (prepare / lower messages / lower
tool config / decode text+usage / decode tool calls / decode
reasoning / decode throttling exception / auth path validation /
SigV4 plumbing) + 2 recorded cassettes against live Bedrock
(`us.amazon.nova-micro-v1:0` in us-east-1) for streaming text and
streaming tool calls.
AGENTS.md: documents the Bedrock auth model, binary cassette format,
and updates the protocol coverage / cassette backlog.
Deps: @smithy/eventstream-codec, @smithy/util-utf8, aws4fetch (~40KB
combined; matches AI SDK's approach).
Add a `providerExecuted: boolean` flag to `tool-call` and `tool-result`
events plus the persisted `ToolResultPart`. When set, the tool runtime
skips client dispatch (the provider already executed the tool) and folds
both events into the assistant message so the next round's history
carries the call + result for context.
Anthropic: decode `server_tool_use` blocks and the three server tool
result block types (`web_search_tool_result`, `code_execution_tool_result`,
`web_fetch_tool_result`) into `tool-call` / `tool-result` events with
`providerExecuted: true`. Round-trip the same parts back into the
provider when the assistant message is replayed in subsequent requests.
Result block error payloads (`*_tool_result_error`) surface as
`result.type === "error"`.
OpenAI Responses: decode hosted tool items emitted via
`response.output_item.done` (`web_search_call`, `file_search_call`,
`code_interpreter_call`, `computer_use_call`, `image_generation_call`,
`mcp_call`, `local_shell_call`) as `tool-call` + `tool-result` pairs
with `providerExecuted: true`. Each tool's input fields are pulled out
explicitly; the full item is passed through as the result payload so
consumers can read outputs / sources / status without re-decoding.
Tool runtime: extend the dispatch decision so provider-executed
tool-calls bypass the handler lookup, and tool-result events with
`providerExecuted: true` are appended to the assistant content for
round-trip rather than being treated as a separate tool message.
Tests: 7 new deterministic fixtures cover Anthropic decode (success +
error result + round-trip + unknown server tool name), OpenAI Responses
decode (web_search_call, code_interpreter_call), and tool-runtime
skip-dispatch.
AGENTS.md updates the runtime section to describe pass-through behavior
and notes the transport-agnostic design that keeps a future WebSocket
adapter (e.g. OpenAI Codex backend) as a sibling rather than a core
rewrite.
Captures both model rounds of the typed ToolRuntime tool loop into a
single multi-interaction cassette: round 1 carries the user prompt and
returns a get_weather tool call; round 2 carries the assistant tool call
plus tool result and returns a final answer.
Verifies the multi-interaction cassette infrastructure end-to-end against
a real provider.
The cassette layer already stored interactions in an array, but replay
always used find-first structural matching and cassettes were written
as one minified JSON line. That makes tool-loop and retry recordings
unworkable: identical requests collapse to one response, and large
recordings are unreadable on review.
- Add `sequentialMatcher` for position-based dispatch so identical
retries map to recorded responses in order via an internal cursor.
- Pretty-print cassette JSON on write and reformat existing fixtures so
multi-interaction diffs stay reviewable.
- Add deterministic `record-replay.test.ts` covering default vs
sequential dispatch and cursor exhaustion.
- Add an OpenAI Chat tool-loop recorded test scaffold gated behind
`OPENAI_API_KEY` so a single `RECORD=true` run captures every
model round of the loop into one cassette file.
- Update AGENTS.md to document multi-interaction cassettes and the
matcher options, and mark the cassette ergonomics TODO complete.
Simplify pass after the typed ToolRuntime initial drop. Findings from a
parallel review (code reuse + quality + perf):
src/tool.ts
- Tool now carries memoized decode/encode codecs and a precomputed
ToolDefinition, derived once at tool() construction time. The runtime no
longer rebuilds Schema closures or JSON Schema docs per call/per run.
- Constrains parameters/success to Schema.Codec<T, any, never, never> so
the codecs have no service requirements. Drops the 'as unknown as' casts
the runtime needed previously.
- Fixes a latent bug: schemas with $ref now correctly emit $defs on
ToolDefinition.inputSchema (toJsonSchemaDocument's definitions were
silently dropped before).
src/tool-runtime.ts
- Uses LLMRequest constructor instead of 'as LLMRequest' casts.
- Default tool dispatch concurrency is 10 (was 'unbounded'); exposed via
RunOptions.concurrency. Unbounded is still available for handlers that
do not share a saturable resource.
- Drops dead 'usage' state, the single-use Dispatched interface, and the
DEFAULT_MAX_STEPS constant per the inline-when-used style rule.
- accumulate() now factors text-delta and reasoning-delta into one helper.
test/lib/openai-chunks.ts (new)
- Shared deltaChunk / usageChunk / toolCallChunk / finishChunk helpers.
test/lib/http.ts
- scriptedResponses moved here from tool-runtime.test.ts so future
multi-step adapter tests can reuse it. Also picks up parallel work that
swapped HandlerInput to a 'respond' callback for cleaner Response
construction.
test/tool-runtime.test.ts
- Uses LLMEvent.guards for typed event filtering instead of cast-and-check.
- Concurrent test now uses sseEvents + deltaChunk instead of a hand-rolled
body string.
Includes parallel callsite updates in test/adapter.test.ts and
test/provider/openai-compatible-chat.test.ts that adopt the 'respond' API
in lib/http.ts.
Schema-first, Effect-first tool loop:
- 'tool({ description, parameters, success, execute })' constructs a fully
typed Tool. parameters and success are Effect Schemas; execute is typed
against them and returns Effect<Success, ToolFailure>. Handler dependencies
are closed over at construction time so the runtime never sees per-tool
services.
- 'ToolRuntime.run(client, { request, tools, maxSteps?, stopWhen? })' streams
the model, decodes tool-call inputs against parameters, dispatches to the
matching handler, encodes results against success, emits tool-result events,
appends assistant + tool messages, and re-streams. Stops on non-tool-calls
finish, maxSteps, or stopWhen.
- Three recoverable error paths emit tool-error events so the model can
self-correct: unknown tool name, input fails parameters Schema, handler
returns ToolFailure. Defects fail the stream.
- 'ToolFailure' added to the schema and exported as the single forced error
channel for handlers.
- Tool definitions on the LLMRequest are derived via toJsonSchemaDocument so
consumers don't write JSON Schema by hand.
8 deterministic fixture tests cover the loop, errors, maxSteps, stopWhen, and
parallel tool calls in one step.
Per the package style guide, sync if/return functions that need to fail
should yield the error directly via Effect.gen rather than ladder
Effect.fail / Effect.succeed across every branch.
Touches all four adapters' tool-choice lowering. The naming-required
validation now reads as 'guard, then return' rather than embedded in a
chain of monadic returns. Behavior unchanged.
Every adapter's parse already produces LLMEvents (via the process callback in
the shared sse helper), and every raise was Stream.make(event). The Chunk type
parameter, the raise field, the RaiseState interface, and the Stream.flatMap
raise step in client.stream were all pure overhead.
- Adapter contract shrinks from <Draft, Target, Chunk> to <Draft, Target>.
- All four adapters drop their raise: (event) => Stream.make(event) line.
- client.stream skips the no-op flatMap.
- AGENTS.md adapter section reflects the simpler contract.
Updates the AGENTS.md TODO list:
- mark Responses, Anthropic, and Gemini adapter coverage as done
- mark the Gemini schema sanitizer port as done
- add concrete next-step items for OpenCode integration: ModelRef bridge,
request bridge, provider-quirk patches, request/stream parity tests, and
a flagged rollout against existing session/llm.test.ts cases
- add OpenAI-compatible Chat, Bedrock Converse, and Vertex routing as
outstanding adapter/dispatch decisions
Kit Langton