Compare commits

...

17 Commits

Author SHA1 Message Date
James Long 8321239461 test(opencode): simplify external directory layer wiring 2026-06-20 18:32:57 -04:00
imranshaiedi-byte 22cc758b1a feat(opencode): expose High/Max thinking variants for GLM-5.2 (#32446)
Co-authored-by: Aiden Cline <aidenpcline@gmail.com>
2026-06-20 16:59:08 -05:00
opencode-agent[bot] 0b7ec51d0a chore: generate 2026-06-20 21:05:57 +00:00
Kit Langton 4f1a9d7aef fix(core): honor configured agent step limits (#33142) 2026-06-20 23:04:30 +02:00
Adam 503309d244 fix(stats): tolerate pending user column 2026-06-20 15:56:52 -05:00
opencode-agent[bot] 1c76587ce2 chore: generate 2026-06-20 19:56:12 +00:00
Adam 24c70ec974 feat(stats): add unique user charts 2026-06-20 14:54:45 -05:00
Lucas Kim 2d993cd0d5 fix(experimental llm pkg): forward topK to Converse via additionalModelRequestFields (#33030) 2026-06-20 14:35:05 -05:00
卫斯李 babe5070e2 fix(opencode): use toLowerCase for Devstral model detection (#33109)
Co-authored-by: Robin1987China <fanrenzhige@163.com>
2026-06-20 14:19:27 -05:00
Dax 009f3799cd refactor(tui): simplify inline tool spacing (#33097) 2026-06-20 09:35:20 -04:00
Adam 95237a90a7 fix(stats): align model peers ranking
deploy / deploy (push) Has been cancelled
2026-06-20 06:14:24 -05:00
Aiden Cline e6cdc543f3 fix(tui): render console org load errors inline (#33040)
deploy / deploy (push) Has been cancelled
nix-eval / nix-eval (push) Has been cancelled
nix-hashes / compute-hash (blacksmith-4vcpu-ubuntu-2404, x86_64-linux) (push) Has been cancelled
nix-hashes / compute-hash (blacksmith-4vcpu-ubuntu-2404-arm, aarch64-linux) (push) Has been cancelled
nix-hashes / compute-hash (macos-15-intel, x86_64-darwin) (push) Has been cancelled
nix-hashes / compute-hash (macos-latest, aarch64-darwin) (push) Has been cancelled
nix-hashes / update-hashes (push) Has been cancelled
publish / version (push) Has been cancelled
publish / build-cli (push) Has been cancelled
publish / sign-cli-windows (push) Has been cancelled
publish / build-electron (map[bun_install_flags:--os=darwin --cpu=arm64 host:macos-26 platform_flag:--mac --arm64 target:aarch64-apple-darwin]) (push) Has been cancelled
publish / build-electron (map[bun_install_flags:--os=darwin --cpu=x64 host:macos-26-intel platform_flag:--mac --x64 target:x86_64-apple-darwin]) (push) Has been cancelled
publish / build-electron (map[host:blacksmith-4vcpu-ubuntu-2404 platform_flag:--linux target:x86_64-unknown-linux-gnu]) (push) Has been cancelled
publish / build-electron (map[host:blacksmith-4vcpu-ubuntu-2404-arm platform_flag:--linux --arm64 target:aarch64-unknown-linux-gnu]) (push) Has been cancelled
publish / build-electron (map[host:blacksmith-4vcpu-windows-2025 platform_flag:--win target:x86_64-pc-windows-msvc]) (push) Has been cancelled
publish / build-electron (map[host:windows-2025 platform_flag:--win --arm64 target:aarch64-pc-windows-msvc]) (push) Has been cancelled
publish / publish (push) Has been cancelled
generate / generate (push) Has been cancelled
typecheck / typecheck (push) Has been cancelled
2026-06-19 19:41:22 -04:00
Aiden Cline f092bafe88 tweak: remove steering wrapper that can bust cache (#33039) 2026-06-19 19:10:09 -04:00
Grant Martin 3f1fffeb6d fix(core): fix command docs in customize-opencode skill (#32718) 2026-06-19 18:22:27 -04:00
Luke Parker c6083a474c test(app): add manual performance diagnostics (#32937) 2026-06-19 19:39:02 +02:00
opencode-agent[bot] 10ec856ff2 chore: generate 2026-06-19 16:38:37 +00:00
Adam 0f6c9b3874 chore(stats): update data seo metadata
deploy / deploy (push) Has been cancelled
2026-06-19 11:36:42 -05:00
58 changed files with 3599 additions and 474 deletions
+5
View File
@@ -1,3 +1,8 @@
## Priorities
- Prioritise, in this order: stability, simplicity, performance.
- Before changing session or timeline code, record a production benchmark baseline and compare it after the change.
## Debugging
- NEVER try to restart the app, or the server process, EVER.
+13
View File
@@ -0,0 +1,13 @@
- Prioritize stability, then simplicity, then measurement overhead.
- Use Playwright for scenario control, isolation, and completion checks.
- Use Chrome Performance traces for generic browser profiling.
- Use Electron `contentTracing` for packaged multi-process profiling.
- Keep custom probes only for product-specific measurements.
- Do not duplicate measurements across the harness, probes, and traces.
- Run benchmarks serially to avoid cross-test contention.
- Run benchmarks against production builds.
- Keep detailed profiling opt-in when it changes workload behavior.
- Preserve raw diagnostic data or use lossless representations.
- Do not enforce machine-dependent performance thresholds.
- Assert scenario completion and metric collection only.
- Keep normal test discovery free of manual benchmarks.
+77
View File
@@ -0,0 +1,77 @@
# Manual app performance suite
The app's high-volume performance diagnostics live under `packages/app/e2e/performance` and are excluded from normal local and CI Playwright discovery. The benchmark config builds the app and serves the production bundle before running scenarios serially.
Run the suite explicitly from `packages/app`:
```sh
bun run test:bench
```
PowerShell:
```powershell
$env:PLAYWRIGHT_WORKERS = "1"
bun run test:bench
```
The suite contains:
- cold and hot session-tab timing
- cached session repaint and mutation tracing
- streaming timeline throughput, RAF-gap, long-task, geometry, and remount diagnostics
All benchmarks import the shared `benchmark` fixture. Pages created through Playwright's `page` fixture automatically capture main-frame navigation history and emit a Chrome trace when `OPENCODE_PERFORMANCE_TRACE_DIR` is set. Benchmarks that need isolated browser contexts use `withBenchmarkPage`, which owns the context and the same diagnostics lifecycle.
New benchmarks should look like normal Playwright tests:
```ts
import { benchmark, expect } from "../benchmark"
benchmark("measures one interaction", async ({ page, report }) => {
// Only scenario-specific setup and interaction belong here.
report({ durationMs: 42 })
})
```
The fixture requires every benchmark to call `report()`, automatically names and closes traces, captures navigation history, attaches that history when a test fails, and emits metrics as a consistent `BENCHMARK` JSON line.
```text
BENCHMARK {"name":"...","context":{"project":"chromium","platform":"darwin"},"metrics":{...}}
```
Every observed page also emits `BENCHMARK_PAGE` with the same run ID, navigation history, and optional trace path before the final status-bearing `BENCHMARK` record. Chrome traces are browser-wide page-lifetime diagnostics; scenario metrics use narrower explicitly named observation windows.
This follows the stack's own guidance: [Electron recommends repeated Chrome DevTools and Chrome Tracing measurement](https://www.electronjs.org/docs/latest/tutorial/performance), [Chrome DevTools recommends Performance recordings for runtime work](https://developer.chrome.com/docs/devtools/performance), and [Playwright uses traces for test debugging rather than renderer profiling](https://playwright.dev/docs/trace-viewer).
These Playwright benchmarks profile the shared app renderer in Chromium. A future packaged Electron benchmark that needs main-process and multi-process attribution should use Electron's official [`contentTracing`](https://www.electronjs.org/docs/latest/api/content-tracing/) API rather than extending this renderer harness with bespoke process instrumentation.
CPU and high-volume visual profiling are disabled by default. Set `TIMELINE_CPU_PROFILE=1` to enable both, or additionally set `TIMELINE_VISUAL_PROFILE=0` for CPU-only profiling.
The streaming scenario's 30x CPU throttle is a deterministic stress profile, not a simulated end-user device.
Benchmarks do not assert machine-dependent performance budgets. Streaming processes 160 deltas by default and reports renderer-observed completion time, throughput, RAF callback-gap distributions, frame-budget equivalents, and long tasks through final geometry settlement. Delta count and delivery batch are included in result context when overridden. These are main-thread callback diagnostics, not compositor presentation or dropped-frame measurements. Visual-only and geometry metrics are `null` when their probes are disabled. Tab metrics describe sampled DOM observations. Assertions verify scenario and metric collection completion. Repeated repaint states are run-length grouped, but every original observation timestamp is retained alongside raw mutation batches and layout shifts.
Committed smoke and regression tests continue to own correctness coverage for pagination, tab paint, context resize, collapse state, and composer spacing.
## Chrome traces
Set `OPENCODE_PERFORMANCE_TRACE_DIR` to emit a standard Chrome DevTools trace for every benchmark page automatically:
```sh
OPENCODE_PERFORMANCE_TRACE_DIR=/tmp/opencode-performance-traces \
bunx playwright test --config e2e/performance/playwright.config.ts \
timeline/session-tab-switch-benchmark.spec.ts
```
The emitted JSON is a standard Chrome trace and can be loaded directly into the Chrome DevTools Performance panel. `devtools-tracing` can optionally inspect it from the command line without adding package scripts or dependencies:
Trace capture mirrors [Puppeteer's official tracing defaults and lifecycle](https://pptr.dev/api/puppeteer.tracing), using Chrome's `ReturnAsStream` transfer mode and failing when Chromium reports trace data loss.
```sh
bunx devtools-tracing stats <trace-path-from-BENCHMARK_PAGE>
```
INP analysis requires a trace with a supported navigation/interaction insight. Selector statistics require a trace captured with `OPENCODE_PERFORMANCE_SELECTOR_TRACE=1`.
`e2e/performance/playwright.uncapped.config.ts` disables Chromium frame-rate limiting for explicit uncapped diagnostics. Native product benchmarks should use the default Playwright configuration.
+144
View File
@@ -0,0 +1,144 @@
import { expect, test as base, type Browser, type Page, type TestInfo } from "@playwright/test"
import { startChromeTrace } from "./chrome-trace"
type BenchmarkFixtures = {
report: (metrics: Record<string, unknown>, context?: Record<string, unknown>) => void
reportState: { payload?: { metrics: Record<string, unknown>; context: Record<string, unknown> } }
benchmarkResult: void
}
export type PerformancePageDiagnostics = {
navigations: string[]
stop: () => Promise<string | undefined>
}
const pages = new WeakMap<Page, PerformancePageDiagnostics>()
export const benchmark = base.extend<BenchmarkFixtures>({
reportState: async ({}, use) => use({}),
report: async ({ reportState }, use) => {
await use((metrics, context = {}) => {
if (reportState.payload) throw new Error("Benchmark reported metrics more than once")
reportState.payload = { metrics, context }
})
},
benchmarkResult: [
async ({ reportState }, use, testInfo) => {
await use()
const missing = !reportState.payload
console.log(
`BENCHMARK ${JSON.stringify({
schemaVersion: 2,
runID: process.env.OPENCODE_PERFORMANCE_RUN_ID,
name: benchmarkName(testInfo),
status: missing ? "failed" : testInfo.status,
expectedStatus: testInfo.expectedStatus,
retry: testInfo.retry,
repeatEachIndex: testInfo.repeatEachIndex,
context: {
project: testInfo.project.name,
platform: process.platform,
...reportState.payload?.context,
},
metrics: reportState.payload?.metrics ?? null,
error: missing ? "Benchmark did not report metrics" : undefined,
})}`,
)
if (missing && testInfo.status === testInfo.expectedStatus)
throw new Error(`Benchmark did not report metrics: ${benchmarkName(testInfo)}`)
},
{ auto: true },
],
page: async ({ page }, use, testInfo) => {
const name = benchmarkName(testInfo)
const diagnostics = await observePerformancePage(page, name)
try {
await use(page)
} finally {
try {
await reportPerformancePage(name, diagnostics, testInfo)
} finally {
if (testInfo.status !== testInfo.expectedStatus) {
await testInfo.attach("performance-navigations", {
body: JSON.stringify(diagnostics.navigations, null, 2),
contentType: "application/json",
})
}
}
}
},
})
function benchmarkName(testInfo: TestInfo) {
return testInfo.titlePath.slice(1).join(" > ")
}
export { expect }
async function observePerformancePage(page: Page, name: string) {
const navigations: string[] = []
const onNavigation = (frame: ReturnType<Page["mainFrame"]>) => {
if (frame === page.mainFrame()) navigations.push(frame.url())
}
page.on("framenavigated", onNavigation)
const stopTrace = await startChromeTrace(page, name).catch((error) => {
page.off("framenavigated", onNavigation)
throw error
})
let stopping: Promise<string | undefined> | undefined
const diagnostics: PerformancePageDiagnostics = {
navigations,
stop() {
page.off("framenavigated", onNavigation)
return (stopping ??= stopTrace?.() ?? Promise.resolve(undefined))
},
}
pages.set(page, diagnostics)
return diagnostics
}
export async function withBenchmarkPage<T>(
browser: Browser,
name: string,
run: (page: Page) => Promise<T>,
testInfo?: TestInfo,
) {
const context = await browser.newContext()
try {
const page = await context.newPage()
const diagnostics = await observePerformancePage(page, name)
try {
return await run(page)
} finally {
await reportPerformancePage(name, diagnostics, testInfo)
}
} finally {
await context.close()
}
}
async function reportPerformancePage(name: string, diagnostics: PerformancePageDiagnostics, testInfo?: TestInfo) {
const trace = await diagnostics.stop()
console.log(
`BENCHMARK_PAGE ${JSON.stringify({
schemaVersion: 2,
runID: process.env.OPENCODE_PERFORMANCE_RUN_ID,
name,
test: testInfo ? benchmarkName(testInfo) : undefined,
retry: testInfo?.retry,
repeatEachIndex: testInfo?.repeatEachIndex,
context: {
platform: process.platform,
trace,
selectorTrace: process.env.OPENCODE_PERFORMANCE_SELECTOR_TRACE === "1",
},
navigations: diagnostics.navigations,
})}`,
)
}
export function benchmarkDiagnostics(page: Page) {
const diagnostics = pages.get(page)
if (!diagnostics) throw new Error("Performance diagnostics are not installed for this page")
return diagnostics
}
@@ -0,0 +1,95 @@
import type { CDPSession, Page } from "@playwright/test"
import path from "node:path"
import { mkdir, open, rename } from "node:fs/promises"
import { Buffer } from "node:buffer"
import { createHash, randomUUID } from "node:crypto"
const categories = [
"-*",
"devtools.timeline",
"v8.execute",
"disabled-by-default-devtools.timeline",
"disabled-by-default-devtools.timeline.frame",
"toplevel",
"blink.console",
"blink.user_timing",
"latencyInfo",
"disabled-by-default-devtools.timeline.stack",
"disabled-by-default-v8.cpu_profiler",
]
export async function startChromeTrace(page: Page, name: string) {
const directory = process.env.OPENCODE_PERFORMANCE_TRACE_DIR
if (!directory) return
const selectors = process.env.OPENCODE_PERFORMANCE_SELECTOR_TRACE === "1"
const file = await prepareChromeTrace(directory, name, selectors)
const session = await page.context().newCDPSession(page)
try {
await session.send("Tracing.start", {
transferMode: "ReturnAsStream",
traceConfig: {
excludedCategories: categories
.filter((category) => category.startsWith("-"))
.map((category) => category.slice(1)),
includedCategories: [
...categories.filter((category) => !category.startsWith("-")),
...(selectors
? ["disabled-by-default-blink.debug", "disabled-by-default-devtools.timeline.invalidationTracking"]
: []),
],
},
})
} catch (error) {
await Promise.allSettled([session.detach()])
throw error
}
let stopping: Promise<string> | undefined
return () =>
(stopping ??= (async () => {
try {
const complete = new Promise<{ stream?: string; dataLossOccurred: boolean }>((resolve) =>
session.once("Tracing.tracingComplete", resolve),
)
await session.send("Tracing.end")
const result = await complete
if (!result.stream) throw new Error(`Chrome trace stream missing: ${file}`)
const partial = `${file}.partial`
await writeProtocolStream(session, result.stream, partial)
if (result.dataLossOccurred) throw new Error(`Chrome trace lost data; partial capture retained: ${partial}`)
await rename(partial, file)
return file
} finally {
await Promise.allSettled([session.detach()])
}
})())
}
export async function prepareChromeTrace(
directory: string,
name: string,
selectors: boolean,
nonce = randomUUID().slice(0, 8),
) {
await mkdir(directory, { recursive: true })
const run = process.env.OPENCODE_PERFORMANCE_RUN_ID ?? "manual"
const hash = createHash("sha256").update(name).digest("hex").slice(0, 8)
return path.join(
directory,
`${run}-${name.replace(/[^a-zA-Z0-9_-]/g, "-")}-${hash}-${nonce}${selectors ? "-selectors" : ""}.json`,
)
}
async function writeProtocolStream(session: CDPSession, handle: string, file: string) {
const output = await open(file, "wx")
try {
while (true) {
const chunk = await session.send("IO.read", { handle })
await output.write(chunk.base64Encoded ? Buffer.from(chunk.data, "base64") : chunk.data)
if (chunk.eof) break
}
} finally {
await Promise.allSettled([output.close(), session.send("IO.close", { handle })])
}
}
@@ -0,0 +1,20 @@
import config from "../../playwright.config"
const port = Number(process.env.PLAYWRIGHT_PORT ?? 3000)
process.env.PLAYWRIGHT_SERVER_PORT = String(port)
process.env.OPENCODE_PERFORMANCE_RUN_ID ??= `${new Date().toISOString().replace(/[:.]/g, "-")}-${process.pid}`
export default {
...config,
testDir: ".",
testIgnore: "unit/**",
outputDir: "../test-results/performance",
fullyParallel: false,
workers: 1,
reporter: [["html", { outputFolder: "../playwright-report/performance", open: "never" }], ["line"]],
webServer: {
...config.webServer,
command: `bun run build && bun run serve -- --host 0.0.0.0 --port ${port} --strictPort`,
reuseExistingServer: false,
},
}
@@ -0,0 +1,13 @@
import config from "./playwright.config"
export default {
...config,
outputDir: "../test-results/performance-uncapped",
reporter: [["html", { outputFolder: "../playwright-report/performance-uncapped", open: "never" }], ["line"]],
use: {
...config.use,
launchOptions: {
args: ["--disable-frame-rate-limit", "--disable-gpu-vsync"],
},
},
}
@@ -0,0 +1,49 @@
import { benchmark, expect } from "../benchmark"
import { expectSessionTitle } from "../../utils/waits"
import { fixture } from "./session-timeline-stress.fixture"
import {
collectCachedRepaintTrace,
compressCachedRepaintTrace,
installCachedRepaintProbe,
waitForCachedRepaintWindow,
} from "./session-tab-repaint-probe"
import { waitForStableTimeline } from "./session-tab-switch-probe"
import {
installStressSessionTabs,
installTimelineSettings,
mockStressTimeline,
stressSessionHref,
} from "./timeline-test-helpers"
benchmark("samples cached session repaint after the click", async ({ page, report }) => {
benchmark.setTimeout(120_000)
await mockStressTimeline(page)
await installStressSessionTabs(page)
await installTimelineSettings(page)
await page.goto(stressSessionHref(fixture.targetID))
await expectSessionTitle(page, fixture.expected.targetTitle)
await waitForStableTimeline(page, fixture.expected.targetMessageIDs.at(-1)!)
await page
.locator(`[data-slot="titlebar-tabs"] a[href="${stressSessionHref(fixture.sourceID)}"]`)
.first()
.click()
await expectSessionTitle(page, fixture.expected.sourceTitle)
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
await installCachedRepaintProbe(page, {
targetHref: stressSessionHref(fixture.targetID),
destination: fixture.messages[fixture.targetID].map((message) => message.info.id),
source: fixture.messages[fixture.sourceID].map((message) => message.info.id),
last: fixture.expected.targetMessageIDs.at(-1)!,
windowMs: 1_000,
})
await page
.locator(`[data-slot="titlebar-tabs"] a[href="${stressSessionHref(fixture.targetID)}"]`)
.first()
.click()
await Promise.all([expectSessionTitle(page, fixture.expected.targetTitle), waitForCachedRepaintWindow(page, 1_000)])
const result = await collectCachedRepaintTrace(page)
report(compressCachedRepaintTrace(result))
expect(result.samples.length).toBeGreaterThan(0)
})
@@ -0,0 +1,251 @@
import type { Page } from "@playwright/test"
type CachedRepaintTrace = {
timeOriginEpochMs: number
startedAtPerformanceMs: number
samples: {
observedAtMs: number
root: number | undefined
scrollTop: number
scrollHeight: number
bottomErrorPx: number | undefined
last: boolean
rows: { key: string | undefined; node: number; top: number; bottom: number }[]
mounted: number
center: string | undefined
destination: string[]
source: string[]
}[]
mutations: { observedAtMs: number; changed: { type: string; node: number }[] }[]
shifts: { occurredAtMs: number; value: number }[]
windowMs: number
running: boolean
stop: () => void
}
export async function installCachedRepaintProbe(
page: Page,
input: { targetHref: string; destination: string[]; source: string[]; last: string; windowMs: number },
) {
await page.evaluate(({ targetHref, destination, source, last, windowMs }) => {
const destinationIDs = new Set(destination)
const sourceIDs = new Set(source)
const nodeIDs = new WeakMap<Node, number>()
let nextNodeID = 1
const id = (node: Node) => {
const current = nodeIDs.get(node)
if (current) return current
nodeIDs.set(node, nextNodeID)
return nextNodeID++
}
const state: CachedRepaintTrace = {
timeOriginEpochMs: performance.timeOrigin,
startedAtPerformanceMs: 0,
samples: [],
mutations: [],
shifts: [],
windowMs,
running: false,
stop: () => {},
}
const recordShifts = (entries: PerformanceEntry[]) => {
if (!state.running) return
state.shifts.push(
...entries
.map((entry) => {
if (
entry.startTime < state.startedAtPerformanceMs ||
entry.startTime > state.startedAtPerformanceMs + state.windowMs
)
return
return {
occurredAtMs: entry.startTime - state.startedAtPerformanceMs,
value: (entry as PerformanceEntry & { value: number }).value,
}
})
.filter((entry): entry is { occurredAtMs: number; value: number } => entry !== undefined),
)
}
const shiftObserver = new PerformanceObserver((entries) => recordShifts(entries.getEntries()))
shiftObserver.observe({ type: "layout-shift" })
const recordMutations = (entries: MutationRecord[]) => {
if (!state.running) return
const observedAtMs = performance.now() - state.startedAtPerformanceMs
if (observedAtMs > state.windowMs) return
const changed = entries.flatMap((entry) => [
...[...entry.addedNodes].map((node) => ({ type: "add", node: id(node) })),
...[...entry.removedNodes].map((node) => ({ type: "remove", node: id(node) })),
])
if (changed.length) state.mutations.push({ observedAtMs, changed })
}
const mutationObserver = new MutationObserver(recordMutations)
mutationObserver.observe(document.documentElement, { childList: true, subtree: true })
state.stop = () => {
recordShifts(shiftObserver.takeRecords())
recordMutations(mutationObserver.takeRecords())
state.running = false
shiftObserver.disconnect()
mutationObserver.disconnect()
}
const sample = () => {
if (!state.running) return
setTimeout(() => {
if (!state.running) return
const observedAtMs = performance.now() - state.startedAtPerformanceMs
if (observedAtMs > state.windowMs) return
const root = [...document.querySelectorAll<HTMLElement>(".scroll-view__viewport")].find((element) =>
element.querySelector("[data-timeline-row]"),
)
if (root) {
const view = root.getBoundingClientRect()
const rows = [...root.querySelectorAll<HTMLElement>("[data-timeline-key]")]
.map((element) => ({
key: element.dataset.timelineKey,
node: id(element),
rect: element.getBoundingClientRect(),
}))
.filter((item) => item.rect.bottom > view.top && item.rect.top < view.bottom)
.map((item) => ({
key: item.key,
node: item.node,
top: item.rect.top - view.top,
bottom: item.rect.bottom - view.top,
}))
const messages = [...root.querySelectorAll<HTMLElement>("[data-message-id]")]
.filter((element) => {
const rect = element.getBoundingClientRect()
return rect.bottom > view.top && rect.top < view.bottom
})
.map((element) => element.dataset.messageId!)
const spacer = root.querySelector<HTMLElement>('[data-timeline-row="bottom-spacer"]')?.getBoundingClientRect()
state.samples.push({
observedAtMs,
root: id(root),
scrollTop: root.scrollTop,
scrollHeight: root.scrollHeight,
bottomErrorPx: spacer ? spacer.bottom - view.bottom : undefined,
last: messages.includes(last),
rows,
mounted: root.querySelectorAll("[data-timeline-key]").length,
center: document
.elementFromPoint(view.left + view.width / 2, view.top + view.height / 2)
?.textContent?.slice(0, 80),
destination: messages.filter((messageID) => destinationIDs.has(messageID)),
source: messages.filter((messageID) => sourceIDs.has(messageID)),
})
} else {
state.samples.push({
observedAtMs,
root: undefined,
scrollTop: 0,
scrollHeight: 0,
bottomErrorPx: undefined,
last: false,
rows: [],
mounted: 0,
center: document.elementFromPoint(innerWidth / 2, innerHeight / 2)?.textContent?.slice(0, 80),
destination: [],
source: [],
})
}
requestAnimationFrame(sample)
}, 0)
}
document.addEventListener(
"click",
(event) => {
const link = event.target instanceof Element ? event.target.closest("a") : undefined
if (link?.getAttribute("href") !== targetHref) return
state.startedAtPerformanceMs = performance.now()
state.running = true
requestAnimationFrame(sample)
},
{ capture: true, once: true },
)
;(window as Window & { __cachedFlash?: CachedRepaintTrace }).__cachedFlash = state
}, input)
}
export function layoutShiftSample(entry: Pick<PerformanceEntry, "startTime"> & { value: number }, started: number) {
if (entry.startTime < started) return
return { occurredAtMs: entry.startTime - started, value: entry.value }
}
export async function waitForCachedRepaintWindow(page: Page, durationMs: number) {
await page.waitForFunction((durationMs) => {
const state = (window as Window & { __cachedFlash?: CachedRepaintTrace }).__cachedFlash
return !!state?.running && performance.now() - state.startedAtPerformanceMs >= durationMs
}, durationMs)
}
export async function collectCachedRepaintTrace(page: Page) {
return page.evaluate(() => {
const state = (window as Window & { __cachedFlash?: CachedRepaintTrace }).__cachedFlash!
state.stop()
return state
})
}
export function summarizeCachedRepaintTrace(trace: CachedRepaintTrace) {
const roots = trace.samples.map((sample) => sample.root)
const bottomErrors = trace.samples.flatMap((sample) =>
sample.bottomErrorPx === undefined ? [] : [Math.abs(sample.bottomErrorPx)],
)
const category = (sample: CachedRepaintTrace["samples"][number]) => {
if (sample.source.length) return "source"
if (sample.root === undefined || sample.rows.length === 0) return "blank"
if (!sample.destination.length) return "unknown"
if (sample.last && Math.abs(sample.bottomErrorPx ?? Infinity) <= 1) return "correct"
return "wrongDestination"
}
return {
samples: trace.samples.length,
durationMs: trace.samples.at(-1)?.observedAtMs ?? 0,
firstSampleObservedMs: trace.samples[0]?.observedAtMs,
firstSampleCorrect: trace.samples[0] ? category(trace.samples[0]) === "correct" : false,
blankSamples: trace.samples.filter((sample) => category(sample) === "blank").length,
sourceSamples: trace.samples.filter((sample) => category(sample) === "source").length,
wrongDestinationSamples: trace.samples.filter((sample) => category(sample) === "wrongDestination").length,
unknownSamples: trace.samples.filter((sample) => category(sample) === "unknown").length,
rootChanges: roots.slice(1).filter((root, index) => root !== roots[index]).length,
mountedMin: trace.samples.length ? Math.min(...trace.samples.map((sample) => sample.mounted)) : 0,
mountedMax: Math.max(...trace.samples.map((sample) => sample.mounted)),
maxBottomErrorPx: Math.max(0, ...bottomErrors),
mutationBatches: trace.mutations.length,
addedNodes: trace.mutations.reduce(
(sum, batch) => sum + batch.changed.filter((change) => change.type === "add").length,
0,
),
removedNodes: trace.mutations.reduce(
(sum, batch) => sum + batch.changed.filter((change) => change.type === "remove").length,
0,
),
layoutShiftValueSum: trace.shifts.reduce((sum, shift) => sum + shift.value, 0),
maxLayoutShiftValue: Math.max(0, ...trace.shifts.map((shift) => shift.value)),
}
}
export function compressCachedRepaintTrace(trace: CachedRepaintTrace) {
const samples: {
observedAtMs: number[]
state: Omit<CachedRepaintTrace["samples"][number], "observedAtMs">
}[] = []
for (const sample of trace.samples) {
const { observedAtMs, ...state } = sample
const previous = samples.at(-1)
if (previous && JSON.stringify(previous.state) === JSON.stringify(state)) {
previous.observedAtMs.push(observedAtMs)
continue
}
samples.push({ observedAtMs: [observedAtMs], state })
}
return {
timeOriginEpochMs: trace.timeOriginEpochMs,
startedAtPerformanceMs: trace.startedAtPerformanceMs,
windowMs: trace.windowMs,
summary: summarizeCachedRepaintTrace(trace),
samples,
mutations: trace.mutations,
shifts: trace.shifts,
}
}
@@ -0,0 +1,79 @@
import type { Page } from "@playwright/test"
import { expectSessionTitle } from "../../utils/waits"
import { benchmark, expect, withBenchmarkPage } from "../benchmark"
import { fixture } from "./session-timeline-stress.fixture"
import { installStressSessionTabs, mockStressTimeline, stressSessionHref } from "./timeline-test-helpers"
import { measureSessionSwitch, waitForStableTimeline } from "./session-tab-switch-probe"
type Result = Awaited<ReturnType<typeof measureSessionSwitch>>
benchmark("benchmarks cold and hot session tab switching", async ({ browser, report }, testInfo) => {
benchmark.setTimeout(180_000)
const results = { cold: [] as Result[], hot: [] as Result[] }
for (const mode of ["cold", "hot"] as const) {
for (let run = 0; run < 5; run++) {
results[mode].push(
await withBenchmarkPage(browser, `session-tab-switch-${mode}-${run}`, (page) => trial(page, mode), testInfo),
)
}
}
report({ results, summary: summarize(results) })
})
async function trial(page: Page, mode: "cold" | "hot") {
await mockStressTimeline(page)
await installStressSessionTabs(page)
if (mode === "hot") {
await page.goto(stressSessionHref(fixture.targetID))
await expectSessionTitle(page, fixture.expected.targetTitle)
await waitForStableTimeline(page, fixture.expected.targetMessageIDs.at(-1)!)
await switchSession(page, fixture.sourceID, fixture.expected.sourceTitle)
} else {
await page.goto(stressSessionHref(fixture.sourceID))
await expectSessionTitle(page, fixture.expected.sourceTitle)
}
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
const destinationIDs = fixture.messages[fixture.targetID].map((message) => message.info.id)
const sourceIDs = fixture.messages[fixture.sourceID].map((message) => message.info.id)
const lastID = fixture.expected.targetMessageIDs.at(-1)!
const href = stressSessionHref(fixture.targetID)
const result = await measureSessionSwitch(page, {
destinationIDs,
sourceIDs,
lastID,
href,
switch: () => switchSession(page, fixture.targetID, fixture.expected.targetTitle),
})
return result
}
function summarize(results: Record<"cold" | "hot", Result[]>) {
const stats = (values: (number | null)[]) => {
const sorted = values.filter((value): value is number => value !== null).sort((a, b) => a - b)
return {
min: sorted[0] ?? null,
median: sorted[Math.floor(sorted.length / 2)] ?? null,
max: sorted.at(-1) ?? null,
missing: values.length - sorted.length,
}
}
return Object.fromEntries(
Object.entries(results).map(([mode, values]) => [
mode,
{
firstDestinationObservedMs: stats(values.map((value) => value.firstDestinationObservedMs)),
firstCorrectObservedMs: stats(values.map((value) => value.firstCorrectObservedMs)),
stableObservedMs: stats(values.map((value) => value.stableObservedMs)),
},
]),
)
}
async function switchSession(page: Page, sessionID: string, title: string) {
const href = stressSessionHref(sessionID)
const tab = page.locator(`[data-slot="titlebar-tabs"] a[href="${href}"]`).first()
await expect(tab).toBeVisible()
await tab.click()
await expectSessionTitle(page, title)
}
@@ -0,0 +1,46 @@
export type SessionSwitchSample = {
observedAtMs: number
destination: string[]
source: string[]
hasVisibleRows: boolean
last: boolean
bottomErrorPx?: number
}
export function classifySessionSwitch(samples: SessionSwitchSample[]) {
const firstDestination = samples.findIndex((sample) => sample.destination.length > 0)
const firstCorrect = samples.findIndex(isCorrectDestination)
const stable = samples.findIndex((_, index) => isStableSessionSwitch(samples.slice(index, index + 3)))
return {
firstDestinationObservedMs: samples[firstDestination]?.observedAtMs ?? null,
firstCorrectObservedMs: samples[firstCorrect]?.observedAtMs ?? null,
stableObservedMs: samples[stable + 2]?.observedAtMs ?? null,
wrongDestinationSamples: samples
.slice(firstDestination)
.filter((sample) => sample.destination.length > 0 && !sample.last).length,
blankSamples: samples.filter((sample) => !sample.hasVisibleRows).length,
unknownSamples: samples.filter(
(sample) => sample.hasVisibleRows && sample.destination.length === 0 && sample.source.length === 0,
).length,
sourceSamples: samples.filter((sample) => sample.source.length > 0).length,
}
}
export function isCorrectDestination(sample: SessionSwitchSample) {
return (
sample.destination.length > 0 &&
sample.source.length === 0 &&
sample.last &&
Math.abs(sample.bottomErrorPx ?? Infinity) <= 1
)
}
export function isStableSessionSwitch(samples: SessionSwitchSample[]) {
return samples.length === 3 && samples.every(isCorrectDestination)
}
export function isStableDestination(samples: Pick<SessionSwitchSample, "last" | "bottomErrorPx">[]) {
return (
samples.length === 3 && samples.every((sample) => sample.last && Math.abs(sample.bottomErrorPx ?? Infinity) <= 1)
)
}
@@ -0,0 +1,152 @@
import { expect, type Page } from "@playwright/test"
import { classifySessionSwitch, isStableDestination, type SessionSwitchSample } from "./session-tab-switch-metrics"
type SessionSwitchProbe = {
samples: SessionSwitchSample[]
stop: () => void
}
async function installSessionSwitchProbe(
page: Page,
input: { destinationIDs: string[]; sourceIDs: string[]; lastID: string; href: string },
) {
await page.evaluate(({ destinationIDs, sourceIDs, lastID, href }) => {
const destination = new Set(destinationIDs)
const source = new Set(sourceIDs)
const samples: SessionSwitchSample[] = []
let started: number | undefined
let running = true
const sample = () => {
if (!running || started === undefined) return
setTimeout(() => {
if (!running || started === undefined) return
const observedAtMs = performance.now() - started
const root = [...document.querySelectorAll<HTMLElement>(".scroll-view__viewport")].find((element) =>
element.querySelector("[data-timeline-row]"),
)
if (root) {
const view = root.getBoundingClientRect()
const visible = [...root.querySelectorAll<HTMLElement>("[data-message-id]")]
.filter((element) => {
const rect = element.getBoundingClientRect()
return rect.bottom > view.top && rect.top < view.bottom
})
.map((element) => element.dataset.messageId!)
const hasVisibleRows = [...root.querySelectorAll<HTMLElement>("[data-timeline-key]")].some((element) => {
const rect = element.getBoundingClientRect()
return rect.bottom > view.top && rect.top < view.bottom
})
const spacer = root.querySelector<HTMLElement>('[data-timeline-row="bottom-spacer"]')?.getBoundingClientRect()
samples.push({
observedAtMs,
destination: visible.filter((id) => destination.has(id)),
source: visible.filter((id) => source.has(id)),
hasVisibleRows,
last: visible.includes(lastID),
bottomErrorPx: spacer ? spacer.bottom - view.bottom : undefined,
})
} else {
samples.push({ observedAtMs, destination: [], source: [], hasVisibleRows: false, last: false })
}
requestAnimationFrame(sample)
}, 0)
}
document.addEventListener(
"click",
(event) => {
const link = event.target instanceof Element ? event.target.closest("a") : undefined
if (link?.getAttribute("href") !== href) return
started = performance.now()
requestAnimationFrame(sample)
},
{ capture: true, once: true },
)
;(window as Window & { __sessionSwitchProbe?: SessionSwitchProbe }).__sessionSwitchProbe = {
samples,
stop: () => {
running = false
},
}
}, input)
}
async function waitForStableSessionSwitch(page: Page) {
await page.waitForFunction(() => {
const samples = (window as Window & { __sessionSwitchProbe?: SessionSwitchProbe }).__sessionSwitchProbe?.samples
if (!samples) return false
return samples.some((_, index) => {
const stable = samples.slice(index, index + 3)
return (
stable.length === 3 &&
stable.every(
(sample) =>
sample.destination.length > 0 &&
sample.source.length === 0 &&
sample.last &&
Math.abs(sample.bottomErrorPx ?? Infinity) <= 1,
)
)
})
})
}
async function collectSessionSwitchResult(page: Page) {
const samples = await page.evaluate(() => {
const probe = (window as Window & { __sessionSwitchProbe?: SessionSwitchProbe }).__sessionSwitchProbe!
probe.stop()
return probe.samples
})
return classifySessionSwitch(samples)
}
export async function measureSessionSwitch(
page: Page,
input: { destinationIDs: string[]; sourceIDs: string[]; lastID: string; href: string; switch: () => Promise<void> },
) {
const { switch: run, ...probe } = input
await installSessionSwitchProbe(page, probe)
await run()
await waitForStableSessionSwitch(page)
return collectSessionSwitchResult(page)
}
export async function waitForStableTimeline(page: Page, lastID: string) {
const samples: Pick<SessionSwitchSample, "last" | "bottomErrorPx">[] = []
await expect
.poll(
async () => {
samples.push(
await page.evaluate(
(lastID) =>
new Promise<Pick<SessionSwitchSample, "last" | "bottomErrorPx">>((resolve) => {
requestAnimationFrame(() =>
setTimeout(() => {
const root = [...document.querySelectorAll<HTMLElement>(".scroll-view__viewport")].find((element) =>
element.querySelector("[data-timeline-row]"),
)
if (!root) {
resolve({ last: false })
return
}
const view = root.getBoundingClientRect()
const last = [...root.querySelectorAll<HTMLElement>("[data-message-id]")].some((element) => {
if (element.dataset.messageId !== lastID) return false
const rect = element.getBoundingClientRect()
return rect.bottom > view.top && rect.top < view.bottom
})
const spacer = root
.querySelector<HTMLElement>('[data-timeline-row="bottom-spacer"]')
?.getBoundingClientRect()
resolve({ last, bottomErrorPx: spacer ? spacer.bottom - view.bottom : undefined })
}, 0),
)
}),
lastID,
),
)
return isStableDestination(samples.slice(-3))
},
{ timeout: 30_000, intervals: [0] },
)
.toBe(true)
}
@@ -0,0 +1,488 @@
import { base64Encode } from "@opencode-ai/core/util/encode"
import type { Page } from "@playwright/test"
import { mockOpenCodeServer } from "../../utils/mock-server"
import { expectAppVisible, expectSessionTitle } from "../../utils/waits"
import { expect } from "../benchmark"
const directory = "C:/OpenCode/TimelineStateRegression"
const projectID = "proj_timeline_state_regression"
const sessionID = "ses_timeline_state_regression"
const userMessageID = "msg_user_regression"
const assistantMessageID = "msg_assistant_regression"
const editPartID = "prt_0001_edit"
export const textPartID = "prt_9999_text"
const title = "Timeline collapse state regression"
const model = { providerID: "opencode", modelID: "claude-opus-4-6", variant: "max" }
type EventPayload = {
directory: string
payload: Record<string, unknown>
}
const userMessage = {
info: {
id: userMessageID,
sessionID,
role: "user",
time: { created: 1700000000000 },
summary: { diffs: [] },
agent: "build",
model,
},
parts: [
{
id: "prt_user_text",
sessionID,
messageID: userMessageID,
type: "text",
text: "Please edit the file.",
},
],
}
const editPart = {
id: editPartID,
sessionID,
messageID: assistantMessageID,
type: "tool",
callID: "call_edit_regression",
tool: "edit",
state: {
status: "completed",
input: { filePath: "src/regression.ts" },
output: "Edited src/regression.ts",
title: "src/regression.ts",
metadata: {
filediff: {
file: "src/regression.ts",
additions: 1,
deletions: 1,
before: "export const value = 'before'\n",
after: "export const value = 'after'\n",
},
diff: "diff --git a/src/regression.ts b/src/regression.ts\n-export const value = 'before'\n+export const value = 'after'\n",
},
time: { start: 1700000001000, end: 1700000002000 },
},
}
const streamedTextPart = {
id: textPartID,
sessionID,
messageID: assistantMessageID,
type: "text",
text: "Streaming added a later assistant text part.",
}
const assistantMessage = {
info: {
id: assistantMessageID,
sessionID,
role: "assistant",
time: { created: 1700000001000 },
parentID: userMessageID,
modelID: model.modelID,
providerID: model.providerID,
mode: "build",
agent: "build",
path: { cwd: directory, root: directory },
cost: 0.01,
tokens: { input: 100, output: 200, reasoning: 0, cache: { read: 0, write: 0 } },
variant: "max",
},
parts: [editPart],
}
export async function setupTimelineBenchmark(page: Page, options: { historyTurns: number; eventBatch: number }) {
const events: EventPayload[] = []
let eventBatch = options.eventBatch
await mockOpenCodeServer(page, {
directory,
project: project(),
provider: provider(),
sessions: [session()],
pageMessages: () => ({
items: [
...Array.from({ length: options.historyTurns }, (_, index) => performanceTurn(index)).flat(),
userMessage,
assistantMessage,
],
}),
events: () => events.splice(0, eventBatch),
eventRetry: 16,
})
await page.addInitScript(() => {
localStorage.setItem(
"settings.v3",
JSON.stringify({
general: {
editToolPartsExpanded: true,
shellToolPartsExpanded: true,
showReasoningSummaries: true,
showSessionProgressBar: true,
},
}),
)
})
await page.setViewportSize({ width: 1366, height: 768 })
const scroller = page.locator(".scroll-view__viewport", { has: page.locator("[data-timeline-row]") })
const text = page.locator(`[data-timeline-part-id="${textPartID}"]`).first()
await page.goto(`/${base64Encode(directory)}/session/${sessionID}`)
await expectSessionTitle(page, title)
await expectAppVisible(scroller)
return {
scroller,
text,
transport: {
enqueue(payload: EventPayload | EventPayload[]) {
events.push(...(Array.isArray(payload) ? payload : [payload]))
},
pendingCount() {
return events.length
},
releaseAll() {
eventBatch = events.length
},
},
async scrollToBottom() {
await scroller.evaluate((element) => {
element.scrollTop = element.scrollHeight
})
},
async waitForStableGeometry() {
await expect
.poll(() => scroller.evaluate((element) => element.scrollHeight - element.clientHeight - element.scrollTop))
.toBeLessThanOrEqual(1)
await page.waitForFunction((partID) => {
const root = [...document.querySelectorAll<HTMLElement>(".scroll-view__viewport")].find((element) =>
element.querySelector(`[data-timeline-part-id="${partID}"]`),
)
if (!root) return false
return new Promise<boolean>((resolve) => {
const height = root.scrollHeight
requestAnimationFrame(() =>
requestAnimationFrame(() =>
resolve(root.scrollHeight === height && root.scrollHeight - root.clientHeight - root.scrollTop <= 1),
),
)
})
}, textPartID)
},
}
}
export function buildInitialStreamEvent(deltaCount: number): EventPayload {
return {
directory,
payload: {
type: "message.part.updated",
properties: {
part: {
...streamedTextPart,
text: `Streaming${streamChunk(0, deltaCount + 1)}\n\n\`\`\`ts\nconst initial = true\n\`\`\``,
},
},
},
}
}
export function buildStreamDeltaEvents(deltaCount: number): EventPayload[] {
return Array.from({ length: deltaCount }, (_, index) => ({
directory,
payload: {
type: "message.part.delta",
properties: {
messageID: assistantMessageID,
partID: textPartID,
field: "text",
delta: streamChunk(index + 1, deltaCount + 1),
},
},
}))
}
function performanceTurn(index: number) {
const suffix = String(index).padStart(4, "0")
const userID = `msg_0000_${suffix}_a_user`
const assistantID = `msg_0000_${suffix}_b_assistant`
const before = historicalSource(index, false)
const after = historicalSource(index, true)
const parts = [
...(index % 5 === 0
? [
{
id: `prt_0000_${suffix}_reasoning`,
sessionID,
messageID: assistantID,
type: "reasoning",
text: `Reviewing the existing implementation. ${"constraint analysis ".repeat(20)}`,
time: { start: 1690000001000 + index * 2_000, end: 1690000001200 + index * 2_000 },
},
]
: []),
{
id: `prt_0000_${suffix}_assistant`,
sessionID,
messageID: assistantID,
type: "text",
text: historicalMarkdown(index),
},
...(index % 8 === 0
? [
{
id: `prt_0000_${suffix}_edit`,
sessionID,
messageID: assistantID,
type: "tool",
callID: `call_0000_${suffix}_edit`,
tool: "edit",
state: {
status: "completed",
input: { filePath: `src/history-${index}.ts` },
output: `Edited src/history-${index}.ts`,
title: `src/history-${index}.ts`,
metadata: {
filediff: { file: `src/history-${index}.ts`, additions: 48, deletions: 48, before, after },
},
time: { start: 1690000001200 + index * 2_000, end: 1690000001400 + index * 2_000 },
},
},
]
: []),
...(index % 12 === 0
? [
{
id: `prt_0000_${suffix}_write`,
sessionID,
messageID: assistantID,
type: "tool",
callID: `call_0000_${suffix}_write`,
tool: "write",
state: {
status: "completed",
input: { filePath: `src/generated-${index}.tsx`, content: after },
output: `Wrote src/generated-${index}.tsx`,
title: `src/generated-${index}.tsx`,
metadata: {
filediff: { file: `src/generated-${index}.tsx`, additions: 32, deletions: 0, before: "", after },
},
time: { start: 1690000001400 + index * 2_000, end: 1690000001500 + index * 2_000 },
},
},
]
: []),
...(index % 16 === 0
? [
{
id: `prt_0000_${suffix}_patch`,
sessionID,
messageID: assistantID,
type: "tool",
callID: `call_0000_${suffix}_patch`,
tool: "apply_patch",
state: {
status: "completed",
input: { patchText: realisticPatch(index) },
output: "Success. Updated src/components/SessionCard.tsx",
title: "src/components/SessionCard.tsx",
metadata: {
files: [
{
filePath: "src/components/SessionCard.tsx",
relativePath: "src/components/SessionCard.tsx",
type: "update",
additions: 8,
deletions: 3,
patch: realisticPatch(index),
before,
after,
},
],
},
time: { start: 1690000001500 + index * 2_000, end: 1690000001700 + index * 2_000 },
},
},
]
: []),
]
return [
{
info: {
id: userID,
sessionID,
role: "user",
time: { created: 1690000000000 + index * 2_000 },
summary: { diffs: [] },
agent: "build",
model,
},
parts: [
{
id: `prt_0000_${suffix}_user`,
sessionID,
messageID: userID,
type: "text",
text: `Historical prompt ${index}`,
},
],
},
{
info: {
id: assistantID,
sessionID,
role: "assistant",
time: { created: 1690000001000 + index * 2_000, completed: 1690000001500 + index * 2_000 },
parentID: userID,
modelID: model.modelID,
providerID: model.providerID,
mode: "build",
agent: "build",
path: { cwd: directory, root: directory },
cost: 0.01,
tokens: { input: 100, output: 200, reasoning: 0, cache: { read: 0, write: 0 } },
variant: "max",
finish: "stop",
},
parts,
},
]
}
function historicalMarkdown(index: number) {
const code = `import { For, Show, createSignal } from "solid-js"
type SessionRow = { id: string; title: string; active: boolean }
export function SessionList(props: { rows: SessionRow[] }) {
const [selected, setSelected] = createSignal<string>()
return (
<section aria-label="Sessions">
<For each={props.rows}>{(row) => (
<button classList={{ active: row.active }} onClick={() => setSelected(row.id)}>
<Show when={selected() === row.id} fallback={row.title}>{row.title.toUpperCase()}</Show>
</button>
)}</For>
</section>
)
}`
return `## Session renderer review ${index}
The active session keeps **semantic row identity** while reconciling measured content. See [Solid documentation](https://docs.solidjs.com/) and the inline \`measureElement(node)\` call.
| Concern | Current behavior | Verification |
| --- | --- | --- |
| streaming | appends Markdown blocks | painted frames |
| geometry | anchors visible rows | DOM coordinates |
| tools | preserves expanded state | keyed remount probe |
> Long sessions combine Markdown, syntax highlighting, tool output, and asynchronously rendered diffs.
${index % 4 === 0 ? `\`\`\`tsx\n${code}\n\`\`\`\n\n\`\`\`bash\nbun typecheck\nbun test --preload ./happydom.ts ./src/pages/session\ngit diff --check\n\`\`\`` : "- preserve the viewport anchor\n- avoid replacing stable Markdown nodes\n- process provider deltas without blocking input"}`
}
function historicalSource(index: number, updated: boolean) {
const method = updated ? "toLocaleUpperCase(props.locale)" : "toUpperCase()"
const limit = updated ? 24 : 20
return `import { createMemo, For } from "solid-js"
type Message = {
id: string
role: "user" | "assistant"
text: string
tokens: { input: number; output: number }
}
export function MessageSummary(props: { messages: Message[]; locale: string }) {
const visible = createMemo(() => props.messages.filter((message) => message.text.trim()).slice(-${limit}))
const total = createMemo(() => visible().reduce((sum, message) => sum + message.tokens.output, 0))
return (
<article data-session-index="${index}">
<header>{total().toLocaleString(props.locale)} output tokens</header>
<For each={visible()}>{(message) => <p data-role={message.role}>{message.text.${method}}</p>}</For>
</article>
)
}
`
}
function realisticPatch(index: number) {
return `*** Begin Patch
*** Update File: src/components/SessionCard.tsx
@@
-const title = props.session.title.toUpperCase()
-const messages = props.messages.slice(-20)
+const title = props.session.title.toLocaleUpperCase(props.locale)
+const messages = props.messages.filter((message) => message.text.trim()).slice(-24)
+const outputTokens = messages.reduce((sum, message) => sum + message.tokens.output, 0)
@@
- <h2>{title}</h2>
+ <h2 data-session-index="${index}">{title}</h2>
+ <span>{outputTokens.toLocaleString(props.locale)} output tokens</span>
*** End Patch`
}
export function streamChunk(index: number, count: number) {
if (index === 0) return `\n\n## Implementation plan\n\nStreaming **bold analysis`
if (index === count - 1)
return `\n\`\`\`\n\n## Verification\n\n- **Typecheck:** passed\n- **Timeline geometry:** stable\n- **Streaming output:** benchmark-complete <!-- stream-${index} -->`
const section = Math.floor(index / 18) + 1
const fragments = [
` continues across three`,
` or four word`,
` provider deltas and`,
` closes in this fragment**. <!-- stream-${index} -->\n\n`,
`| Concern | State`,
` | Verification |\n|`,
` --- | ---`,
` | --- |\n|`,
` markdown | incremental |`,
` painted frames | <!-- stream-${index} -->\n\n`,
`\`\`\`tsx\nconst row: SessionRow`,
` = rows[index] ??`,
` fallback\nconst title =`,
` row.title.toLocaleUpperCase(locale)\n`,
`const selected = createMemo(()`,
` => row.id ===`,
` activeID()) // stream-${index}\n`,
`// stream-${index}\n\`\`\`\n\n### Iteration ${section}\n\nStreaming **bold analysis`,
]
return fragments[(index - 1) % fragments.length]!
}
function project() {
return {
id: projectID,
worktree: directory,
vcs: "git",
name: "timeline-state-regression",
time: { created: 1700000000000, updated: 1700000000000 },
sandboxes: [],
}
}
function session() {
return {
id: sessionID,
slug: "timeline-state-regression",
projectID,
directory,
title,
version: "dev",
time: { created: 1700000000000, updated: 1700000000000 },
}
}
function provider() {
return {
all: [
{
id: "opencode",
name: "OpenCode",
models: { "claude-opus-4-6": { id: "claude-opus-4-6", name: "Claude Opus 4.6", limit: { context: 200_000 } } },
},
],
connected: ["opencode"],
default: { providerID: "opencode", modelID: "claude-opus-4-6" },
}
}
@@ -0,0 +1,85 @@
import { benchmark, benchmarkDiagnostics, expect } from "../benchmark"
import {
buildInitialStreamEvent,
buildStreamDeltaEvents,
setupTimelineBenchmark,
textPartID,
} from "./session-timeline-benchmark.fixture"
import { startTimelineProfile } from "./session-timeline-profile"
import {
collectTimelineStreamMetrics,
installTimelineStreamProbe,
startTimelineStreamProbe,
} from "./session-timeline-stream-probe"
benchmark.describe("performance: session timeline streaming", () => {
benchmark("streams assistant text without remounting or oscillating", async ({ page, report }) => {
benchmark.setTimeout(480_000)
const cpuThrottle = Number(process.env.TIMELINE_CPU_THROTTLE ?? 30)
const deltaCount = Number(process.env.TIMELINE_DELTA_COUNT ?? 160)
const historyTurns = Number(process.env.TIMELINE_HISTORY_TURNS ?? 320)
const eventBatch = Number(process.env.TIMELINE_EVENT_BATCH ?? 1)
const minimal = process.env.TIMELINE_MINIMAL === "1"
const profileCPU = process.env.TIMELINE_CPU_PROFILE === "1"
const profileVisual = !minimal && profileCPU && process.env.TIMELINE_VISUAL_PROFILE !== "0"
const fixture = await setupTimelineBenchmark(page, {
historyTurns,
eventBatch,
})
fixture.transport.enqueue(buildInitialStreamEvent(deltaCount))
const contentStart = performance.now()
await expect(fixture.text).toBeVisible()
await expect(fixture.text).toContainText("Implementation plan")
const initialContentObservedMs = performance.now() - contentStart
await fixture.scrollToBottom()
await fixture.waitForStableGeometry()
const profile = await startTimelineProfile(page, { cpuThrottle, profileCPU })
await installTimelineStreamProbe(page, { textPartID, finalIndex: deltaCount, profileVisual, minimal })
const deltas = buildStreamDeltaEvents(deltaCount)
await startTimelineStreamProbe(page)
fixture.transport.enqueue(deltas)
await page.waitForFunction(
(finalIndex) =>
(
window as Window & {
__timelineStreamBenchmark?: { applied: { index: number }[] }
}
).__timelineStreamBenchmark?.applied.some((value) => value.index === finalIndex),
deltaCount,
{ timeout: 420_000 },
)
await expect(fixture.text).toContainText("benchmark-complete")
await expect(fixture.text).toContainText("Streaming")
await fixture.waitForStableGeometry()
const metrics = await collectTimelineStreamMetrics(page, {
textPartID,
finalIndex: deltaCount,
navigations: benchmarkDiagnostics(page).navigations,
})
const delivered = deltas.length - fixture.transport.pendingCount()
await profile.stop()
report(
{
endToEndInitialContentObservedMs: initialContentObservedMs,
...metrics,
deliveredDeltas: delivered,
pendingDeltas: fixture.transport.pendingCount(),
},
{
cpuThrottle,
profileCPU,
profileVisual,
minimal,
queuedDeltas: deltas.length,
historyTurns,
eventBatch,
},
)
await profile.reset()
})
})
@@ -0,0 +1,40 @@
import type { CDPSession, Page } from "@playwright/test"
export async function startTimelineProfile(page: Page, options: { cpuThrottle: number; profileCPU: boolean }) {
const cdp = await page.context().newCDPSession(page)
if (options.cpuThrottle > 1) await cdp.send("Emulation.setCPUThrottlingRate", { rate: options.cpuThrottle })
if (options.profileCPU) {
await cdp.send("Profiler.enable")
await cdp.send("Profiler.setSamplingInterval", { interval: 100 })
await cdp.send("Profiler.start")
}
return {
async stop() {
if (!options.profileCPU) return
const result = await cdp.send("Profiler.stop")
const self = new Map<number, number>()
result.profile.samples?.forEach((id, index) => {
const duration = (result.profile.timeDeltas?.[index] ?? 0) / 1_000
self.set(id, (self.get(id) ?? 0) + duration)
})
console.log(
"timeline cpu profile",
JSON.stringify(
result.profile.nodes
.map((node) => ({
function: node.callFrame.functionName || "(anonymous)",
url: node.callFrame.url,
line: node.callFrame.lineNumber + 1,
selfMs: self.get(node.id) ?? 0,
}))
.filter((node) => node.selfMs > 1)
.sort((a, b) => b.selfMs - a.selfMs)
.slice(0, 40),
),
)
},
async reset() {
if (options.cpuThrottle > 1) await cdp.send("Emulation.setCPUThrottlingRate", { rate: 1 })
},
}
}
@@ -0,0 +1,547 @@
import type { Page } from "@playwright/test"
const STREAM_MARKER_PATTERN = "stream-(\\d+)"
const STREAM_FRAGMENT_COUNT = 18
type TimelineProbeState = {
started: number
ended: number
profileVisual: boolean
minimal: boolean
frames: number[]
frameAt: number[]
applied: { at: number; index: number }[]
geometry: {
scrollTop: number
scrollHeight: number
clientHeight: number
distance: number
virtualHeight: number
headerHeight: number
}[]
blanks: number
longTasks: number[]
layoutShifts: number[]
visibleMounts: number
visibleUnmounts: number
visibleRows: Set<Element>
visibleSubtreeMounts: string[]
visibleSubtreeUnmounts: string[]
visibleSubtreeReplacements: number
visibleSubtreeDropouts: string[]
visibleSubtrees: Map<string, Element>
subtreeKeys: WeakMap<Element, string>
maxOverlap: number
maxGap: number
maxPartTopMovement: number
previousPartTop: number
slowFrames: {
duration: number
index: number
phase: "stream" | "boundary" | "complete" | "unknown"
tokenSpans: number
blocks: number
codeBlocks: number
height: number
distance: number
}[]
scroll: {
calls: number
callNoops: number
sameFrameCalls: number
assignments: number
assignmentNoops: number
lastCallFrame: number
frame: number
}
row: HTMLElement
markdown: HTMLElement
running: boolean
previous: number
cleanup: () => void
start: () => void
}
export async function installTimelineStreamProbe(
page: Page,
options: { textPartID: string; finalIndex: number; profileVisual: boolean; minimal: boolean },
) {
await page.evaluate(
({ textPartID, finalIndex, profileVisual, minimal, markerPattern, fragmentCount }) => {
const part = document.querySelector<HTMLElement>(`[data-timeline-part-id="${textPartID}"]`)
const row = part?.closest<HTMLElement>("[data-timeline-row]")
const markdown = part?.querySelector<HTMLElement>('[data-component="markdown"]')
const root = part?.closest<HTMLElement>(".scroll-view__viewport")
if (!part || !row || !markdown || !root) throw new Error("missing streaming benchmark nodes")
const viewport = root.getBoundingClientRect()
const state: TimelineProbeState = {
started: 0,
ended: Infinity,
profileVisual,
minimal,
frames: [],
frameAt: [],
applied: [],
geometry: [],
blanks: 0,
longTasks: [],
layoutShifts: [],
visibleMounts: 0,
visibleUnmounts: 0,
visibleRows: new Set(
[...root.querySelectorAll("[data-timeline-key]")].filter((element) => {
const rect = element.getBoundingClientRect()
return rect.bottom > viewport.top && rect.top < viewport.bottom
}),
),
visibleSubtreeMounts: [],
visibleSubtreeUnmounts: [],
visibleSubtreeReplacements: 0,
visibleSubtreeDropouts: [],
visibleSubtrees: new Map<string, Element>(),
subtreeKeys: new WeakMap<Element, string>(),
maxOverlap: 0,
maxGap: 0,
maxPartTopMovement: 0,
previousPartTop: part.getBoundingClientRect().top,
slowFrames: [],
scroll: {
calls: 0,
callNoops: 0,
sameFrameCalls: 0,
assignments: 0,
assignmentNoops: 0,
lastCallFrame: -1,
frame: 0,
},
row,
markdown,
running: false,
previous: 0,
cleanup: () => {},
start: () => {},
}
;(window as Window & { __timelineStreamBenchmark?: TimelineProbeState }).__timelineStreamBenchmark = state
const scrollTo = Element.prototype.scrollTo
const scrollTop = Object.getOwnPropertyDescriptor(Element.prototype, "scrollTop")!
if (profileVisual) {
Element.prototype.scrollTo = function (...args) {
state.scroll.calls += 1
const top = typeof args[0] === "object" ? args[0]?.top : args[1]
if (typeof top === "number") {
const target = Math.min(top, this.scrollHeight - this.clientHeight)
if (Math.abs(this.scrollTop - target) < 1) state.scroll.callNoops += 1
}
if (state.scroll.lastCallFrame === state.scroll.frame) state.scroll.sameFrameCalls += 1
state.scroll.lastCallFrame = state.scroll.frame
return scrollTo.apply(this, args)
}
Object.defineProperty(Element.prototype, "scrollTop", {
configurable: true,
get: scrollTop.get,
set(value) {
state.scroll.assignments += 1
if (Math.abs(this.scrollTop - value) < 1) state.scroll.assignmentNoops += 1
scrollTop.set!.call(this, value)
},
})
}
const recordLongTasks = (entries: PerformanceEntry[]) => {
if (!state.running) return
state.longTasks.push(
...entries
.filter((entry) => entry.startTime >= state.started && entry.startTime <= state.ended)
.map((entry) => entry.duration),
)
}
const longTaskObserver = new PerformanceObserver((list) => recordLongTasks(list.getEntries()))
longTaskObserver.observe({ type: "longtask" })
const recordLayoutShifts = (entries: PerformanceEntry[]) => {
if (!state.running) return
state.layoutShifts.push(
...entries
.map((entry) => {
const shift = entry as LayoutShiftEntry
if (shift.startTime < state.started || shift.hadRecentInput) return
return shift.value
})
.filter((value): value is number => value !== undefined),
)
}
const layoutShiftObserver = profileVisual
? new PerformanceObserver((list) => recordLayoutShifts(list.getEntries()))
: undefined
layoutShiftObserver?.observe({ type: "layout-shift", buffered: true })
const visible = (element: Element) => {
const rect = element.getBoundingClientRect()
const viewport = root.getBoundingClientRect()
const style = getComputedStyle(element)
return (
element.isConnected &&
rect.width > 0 &&
rect.height > 0 &&
rect.bottom > viewport.top &&
rect.top < viewport.bottom &&
style.display !== "none" &&
style.visibility !== "hidden" &&
Number(style.opacity) > 0
)
}
const critical = [
"[data-timeline-part-id]",
'[data-component="edit-content"]',
'[data-component="apply-patch-file-diff"]',
'[data-component="file"]',
'[data-component="markdown-code"]',
"[data-markdown-block]",
].join(",")
const describe = (element: Element) => {
const cached = state.subtreeKeys.get(element)
if (!element.isConnected && cached) return cached
const part = element.closest<HTMLElement>("[data-timeline-part-id]")?.dataset.timelinePartId ?? "unknown"
const block = element
.closest<HTMLElement>("[data-markdown-key]")
?.dataset.markdownKey?.replace(/:(?:code|full|live)$/, "")
const component =
element.getAttribute("data-component") ?? element.getAttribute("data-markdown-block") ?? element.tagName
const key = `${part}:${block ?? "root"}:${component}`
state.subtreeKeys.set(element, key)
return key
}
const recordMutations = (records: MutationRecord[]) => {
if (!state.running) return
records.forEach((record) => {
record.addedNodes.forEach((node) => {
if (node instanceof HTMLElement && node.matches("[data-timeline-key]") && visible(node)) {
state.visibleMounts += 1
state.visibleRows.add(node)
}
if (!(node instanceof Element)) return
const added = [node, ...node.querySelectorAll(critical)].filter((element) => element.matches(critical))
added.forEach((element) => {
if (visible(element)) state.visibleSubtreeMounts.push(describe(element))
})
})
record.removedNodes.forEach((node) => {
if (node instanceof HTMLElement && node.matches("[data-timeline-key]") && state.visibleRows.delete(node))
state.visibleUnmounts += 1
if (!(node instanceof Element)) return
const removed = [node, ...node.querySelectorAll(critical)].filter((element) => element.matches(critical))
removed.forEach((element) => {
const key = describe(element)
if (state.visibleSubtrees.get(key) === element) state.visibleSubtreeUnmounts.push(key)
})
})
})
}
const mutationObserver = profileVisual ? new MutationObserver(recordMutations) : undefined
mutationObserver?.observe(root, { childList: true, subtree: true })
const currentPart = () => root.querySelector<HTMLElement>(`[data-timeline-part-id="${textPartID}"]`)
const observeProgress = (at: number) => {
if (!state.running) return
const content = currentPart()?.textContent ?? ""
const index = content.includes("benchmark-complete")
? finalIndex
: Number(content.match(new RegExp(markerPattern, "g"))?.at(-1)?.match(/\d+/)?.[0] ?? -1)
if (index >= 0 && index !== state.applied.at(-1)?.index) state.applied.push({ at, index })
}
const progressObserver = new MutationObserver(() => observeProgress(performance.now()))
progressObserver.observe(root, { characterData: true, childList: true, subtree: true })
state.cleanup = () => {
recordLongTasks(longTaskObserver.takeRecords())
recordLayoutShifts(layoutShiftObserver?.takeRecords() ?? [])
recordMutations(mutationObserver?.takeRecords() ?? [])
if (progressObserver.takeRecords().length) observeProgress(performance.now())
longTaskObserver.disconnect()
layoutShiftObserver?.disconnect()
mutationObserver?.disconnect()
progressObserver.disconnect()
if (!profileVisual) return
Element.prototype.scrollTo = scrollTo
Object.defineProperty(Element.prototype, "scrollTop", scrollTop)
}
const sample = (now: number) => {
if (!state.running) return
state.frameAt.push(now)
observeProgress(now)
if (minimal) {
state.frames.push(now - state.previous)
state.previous = now
requestAnimationFrame(sample)
return
}
setTimeout(() => {
if (!state.running) return
state.scroll.frame += 1
const duration = now - state.previous
state.frames.push(duration)
state.previous = now
const virtualRoot = root.querySelector<HTMLElement>("[data-timeline-virtual-content]")
const header = root.querySelector<HTMLElement>("[data-session-title]")
state.geometry.push({
scrollTop: root.scrollTop,
scrollHeight: root.scrollHeight,
clientHeight: root.clientHeight,
distance: root.scrollHeight - root.clientHeight - root.scrollTop,
virtualHeight: virtualRoot?.getBoundingClientRect().height ?? 0,
headerHeight: header?.getBoundingClientRect().height ?? 0,
})
const viewport = root.getBoundingClientRect()
if (profileVisual) {
const visibleRows = [...root.querySelectorAll<HTMLElement>("[data-timeline-key]")]
.map((element) => ({ element, rect: element.getBoundingClientRect() }))
.filter((item) => item.rect.bottom > viewport.top && item.rect.top < viewport.bottom)
.sort((a, b) => a.rect.top - b.rect.top)
state.visibleRows = new Set(visibleRows.map((item) => item.element))
const rows = visibleRows.map((item) => item.rect)
rows.slice(1).forEach((rect, index) => {
const previous = rows[index]!
state.maxOverlap = Math.max(state.maxOverlap, previous.bottom - rect.top)
state.maxGap = Math.max(state.maxGap, rect.top - previous.bottom)
})
const partTop = part.getBoundingClientRect().top
state.maxPartTopMovement = Math.max(state.maxPartTopMovement, Math.abs(partTop - state.previousPartTop))
state.previousPartTop = partTop
}
const visibleRow = [...root.querySelectorAll<HTMLElement>("[data-timeline-row]")].some((element) => {
const rect = element.getBoundingClientRect()
return rect.bottom > viewport.top && rect.top < viewport.bottom
})
if (!visibleRow) state.blanks += 1
if (profileVisual) {
const subtrees = new Map<string, { element: Element; rendered: boolean }>()
const visibleSubtrees = new Map<string, Element>()
root.querySelectorAll(critical).forEach((element) => {
const key = describe(element)
const rect = element.getBoundingClientRect()
const style = getComputedStyle(element)
const rendered =
element.isConnected &&
rect.width > 0 &&
rect.height > 0 &&
style.display !== "none" &&
style.visibility !== "hidden" &&
Number(style.opacity) > 0
subtrees.set(key, { element, rendered })
if (rendered && rect.bottom > viewport.top && rect.top < viewport.bottom) {
const previous = state.visibleSubtrees.get(key)
if (previous && previous !== element && key.startsWith(`${textPartID}:`))
state.visibleSubtreeReplacements += 1
visibleSubtrees.set(key, element)
}
})
state.visibleSubtrees.forEach((element, key) => {
const current = subtrees.get(key)
if (key.startsWith(`${textPartID}:`) && !current?.rendered) {
const markdown = part.querySelector<HTMLElement>('[data-component="markdown"]')
state.visibleSubtreeDropouts.push(
`${key}:projection=${markdown?.dataset.markdownProjectionLength}/${markdown?.dataset.markdownProjectionBlocks}:result=${markdown?.dataset.markdownResultLength}/${markdown?.dataset.markdownResultBlocks}:applied=${markdown?.dataset.markdownAppliedBlocks}:dom=${markdown?.children.length}`,
)
}
if (element.matches('[data-component="file"]')) {
const hadLines = element.hasAttribute("data-profiler-had-lines")
const hasLines = element.shadowRoot?.querySelector("[data-line]") != null
if (hasLines) element.setAttribute("data-profiler-had-lines", "")
if (hadLines && !hasLines) state.visibleSubtreeDropouts.push(`${key}:shadow-lines`)
}
})
state.visibleSubtrees = visibleSubtrees
}
if (profileVisual && duration > 33.34) {
const livePart = currentPart()
const content = livePart?.textContent ?? ""
const complete = content.includes("benchmark-complete")
const index = complete
? finalIndex
: Number(content.match(new RegExp(markerPattern, "g"))?.at(-1)?.match(/\d+/)?.[0] ?? -1)
state.slowFrames.push({
duration,
index,
phase: complete
? "complete"
: index >= 0 && index % fragmentCount === 0
? "boundary"
: index >= 0
? "stream"
: "unknown",
tokenSpans: livePart?.querySelectorAll(".shiki span").length ?? 0,
blocks: livePart?.querySelectorAll("[data-markdown-block]").length ?? 0,
codeBlocks: livePart?.querySelectorAll('[data-component="markdown-code"]').length ?? 0,
height: livePart?.getBoundingClientRect().height ?? 0,
distance: root.scrollHeight - root.clientHeight - root.scrollTop,
})
}
requestAnimationFrame(sample)
}, 0)
}
state.start = () => {
state.started = performance.now()
state.previous = state.started
state.running = true
requestAnimationFrame(sample)
}
},
{ ...options, markerPattern: STREAM_MARKER_PATTERN, fragmentCount: STREAM_FRAGMENT_COUNT },
)
}
export function startTimelineStreamProbe(page: Page) {
return page.evaluate(() => {
const state = (window as Window & { __timelineStreamBenchmark?: TimelineProbeState }).__timelineStreamBenchmark
if (!state) throw new Error("missing streaming benchmark state")
state.start()
})
}
type LayoutShiftEntry = PerformanceEntry & { value: number; hadRecentInput?: boolean }
export function layoutShiftValue(
entry: Pick<LayoutShiftEntry, "startTime" | "value" | "hadRecentInput">,
start: number,
) {
if (entry.startTime < start || entry.hadRecentInput) return
return entry.value
}
export function removeVisibleRow<T>(visible: Set<T>, row: T) {
return visible.delete(row)
}
export function streamProgress(content: string) {
const index = Number(content.match(new RegExp(STREAM_MARKER_PATTERN, "g"))?.at(-1)?.match(/\d+/)?.[0] ?? -1)
return {
index,
phase: content.includes("benchmark-complete")
? ("complete" as const)
: index >= 0 && index % STREAM_FRAGMENT_COUNT === 0
? ("boundary" as const)
: index >= 0
? ("stream" as const)
: ("unknown" as const),
}
}
export async function collectTimelineStreamMetrics(
page: Page,
options: { textPartID: string; finalIndex: number; navigations: string[] },
) {
return page.evaluate(({ textPartID, finalIndex, navigations }) => {
const state = (window as Window & { __timelineStreamBenchmark?: TimelineProbeState }).__timelineStreamBenchmark
if (!state) throw new Error(`missing streaming benchmark state after navigation: ${JSON.stringify(navigations)}`)
state.ended = performance.now()
state.cleanup()
state.running = false
const part = document.querySelector<HTMLElement>(`[data-timeline-part-id="${textPartID}"]`)
const row = part?.closest<HTMLElement>("[data-timeline-row]")
const markdown = part?.querySelector<HTMLElement>('[data-component="markdown"]')
const sorted = state.frames.slice().sort((a, b) => a - b)
const duration = state.frames.reduce((sum, value) => sum + value, 0)
const longestSlowStreak = state.frames.reduce(
(result, value) => {
const current = value > 33.34 ? result.current + 1 : 0
return { current, longest: Math.max(result.longest, current) }
},
{ current: 0, longest: 0 },
).longest
const busyStart = state.applied.at(0)?.at
const completion = state.applied.find((value) => value.index === finalIndex)
const busyEnd = completion?.at
const busyFrames =
busyStart === undefined || busyEnd === undefined
? []
: state.frames.filter((_, index) => state.frameAt[index]! >= busyStart && state.frameAt[index]! <= busyEnd)
const busySorted = busyFrames.slice().sort((a, b) => a - b)
const busyDuration = busyFrames.reduce((sum, value) => sum + value, 0)
const completionObservedMs = (completion?.at ?? NaN) - state.started
const visual = state.profileVisual
? {
layoutShiftValueSum: state.layoutShifts.reduce((sum, value) => sum + value, 0),
maxLayoutShiftValue: Math.max(0, ...state.layoutShifts),
visibleMounts: state.visibleMounts,
visibleUnmounts: state.visibleUnmounts,
visibleSubtreeMounts: state.visibleSubtreeMounts,
visibleSubtreeUnmounts: [...new Set(state.visibleSubtreeUnmounts)],
visibleSubtreeReplacements: state.visibleSubtreeReplacements,
visibleSubtreeDropouts: [...new Set(state.visibleSubtreeDropouts)],
maxOverlapPx: state.maxOverlap,
maxGapPx: state.maxGap,
maxPartTopMovementPx: state.maxPartTopMovement,
slowestRafGaps: state.slowFrames
.sort((a, b) => b.duration - a.duration)
.slice(0, 20)
.map((frame) => ({
durationMs: frame.duration,
index: frame.index,
phase: frame.phase,
tokenSpans: frame.tokenSpans,
blocks: frame.blocks,
codeBlocks: frame.codeBlocks,
heightPx: frame.height,
distancePx: frame.distance,
})),
slowRafGapPhases: Object.fromEntries(
["stream", "boundary", "complete", "unknown"].map((phase) => {
const frames = state.slowFrames.filter((frame) => frame.phase === phase)
return [
phase,
{
count: frames.length,
totalMs: frames.reduce((sum, frame) => sum + frame.duration, 0),
maxMs: Math.max(0, ...frames.map((frame) => frame.duration)),
},
]
}),
),
scroll: state.scroll,
}
: null
const geometry = state.minimal
? null
: {
maxDistancePx: Math.max(0, ...state.geometry.map((sample) => sample.distance)),
finalDistancePx: state.geometry.at(-1)?.distance ?? 0,
final: state.geometry.at(-1),
distanceTransitionsPx: state.geometry
.map((sample) => Math.round(sample.distance))
.filter((value, index, values) => index === 0 || value !== values[index - 1]),
bottomDriftTransitions: state.geometry.slice(1).filter((value, index) => {
const previous = state.geometry[index]?.distance ?? 0
return previous <= 1 && value.distance > 1
}).length,
blankSamples: state.blanks,
}
return {
capabilities: { visual: state.profileVisual, geometry: !state.minimal },
completionObservedMs,
deltasPerSecond: Number.isFinite(completionObservedMs) ? finalIndex / (completionObservedMs / 1_000) : null,
rafGapSamples: state.frames.length,
rafCallbackRate: duration ? (state.frames.length * 1000) / duration : 0,
observedProgressWindowRafCallbackRate: busyDuration ? (busyFrames.length * 1000) / busyDuration : null,
observedProgressWindowRafGapP95Ms: busySorted[Math.floor(busySorted.length * 0.95)] ?? null,
observedProgressWindowRafGaps: busyFrames.length,
maxObservedProgressIndex: Math.max(-1, ...state.applied.map((value) => value.index)),
observedProgressTransitions: state.applied.length,
rafGapP50Ms: sorted[Math.floor(sorted.length * 0.5)] ?? 0,
rafGapP95Ms: sorted[Math.floor(sorted.length * 0.95)] ?? 0,
rafGapP99Ms: sorted[Math.floor(sorted.length * 0.99)] ?? 0,
maxRafGapMs: sorted.at(-1) ?? 0,
rafGapsOver33Ms: state.frames.filter((value) => value > 33.34).length,
rafGapsOver50Ms: state.frames.filter((value) => value > 50).length,
missedFrameBudgetEquivalents: state.frames.reduce(
(sum, value) => sum + Math.max(0, Math.round(value / 16.67) - 1),
0,
),
longestRafGapOver33MsStreak: longestSlowStreak,
longTaskCount: state.longTasks.length,
longTaskTimeMs: state.longTasks.reduce((sum, value) => sum + value, 0),
visual,
geometry,
rowReplaced: row !== state.row,
markdownReplaced: markdown !== state.markdown,
domTextCharacters: part?.textContent?.length ?? 0,
}
}, options)
}
@@ -0,0 +1,335 @@
const words = [
"alpha",
"bravo",
"charlie",
"delta",
"echo",
"foxtrot",
"golf",
"hotel",
"india",
"juliet",
"kilo",
"lima",
"metro",
"nova",
"orbit",
"pixel",
"quartz",
"river",
"signal",
"vector",
]
const sourceID = "ses_smoke_source"
const targetID = "ses_smoke_target"
const directory = "C:/OpenCode/SmokeProject"
const projectID = "proj_smoke_timeline"
const model = { providerID: "opencode", modelID: "claude-opus-4-6", variant: "max" }
type MessageInfo = Record<string, unknown> & { id: string; role: "user" | "assistant" }
type MessagePart = Record<string, unknown> & { id: string; type: string; text?: string; tool?: string }
type Message = { info: MessageInfo; parts: MessagePart[] }
function lorem(seed: number, length: number) {
let out = ""
let i = seed
while (out.length < length) {
const word = words[i % words.length]
out += (out ? " " : "") + word
if (i % 17 === 0) out += ".\n\n"
i += 7
}
return out.slice(0, length)
}
function id(prefix: string, value: number) {
return `${prefix}_smoke_${String(value).padStart(4, "0")}`
}
function userMessage(sessionID: string, index: number, textLength: number, diffs: unknown[] = []): Message {
const messageID = id("msg_user", index)
return {
info: {
id: messageID,
sessionID,
role: "user",
time: { created: 1700000000000 + index * 10_000 },
summary: { diffs },
agent: "build",
model,
},
parts: [
{
id: id("prt_user_text", index),
sessionID,
messageID,
type: "text",
text: lorem(index, textLength),
},
],
}
}
function assistantMessage(sessionID: string, index: number, parentID: string, parts: MessagePart[]): Message {
const messageID = id("msg_assistant", index)
return {
info: {
id: messageID,
sessionID,
role: "assistant",
time: { created: 1700000000000 + index * 10_000 + 1_000, completed: 1700000000000 + index * 10_000 + 8_000 },
parentID,
modelID: model.modelID,
providerID: model.providerID,
mode: "build",
agent: "build",
path: { cwd: directory, root: directory },
cost: 0.01,
tokens: { input: 100, output: 200, reasoning: 0, cache: { read: 0, write: 0 } },
variant: "max",
finish: "stop",
},
parts: parts.map((part) => ({
...part,
sessionID,
messageID,
})),
}
}
function textPart(index: number, partIndex: number, length: number): MessagePart {
const prose = lorem(index * 13 + partIndex, length)
const text =
index % 12 === 0
? `${prose}\n\n\`\`\`ts\n${code(index, 80)}\n\`\`\``
: index % 5 === 0
? `${prose}\n\n\`\`\`ts\nexport const value = "${lorem(index, 220)}"\n\`\`\``
: index % 7 === 0
? `${prose}\n\nThe wrapped inline value is \`${lorem(index, 180)}\`.`
: prose
return { id: id(`prt_text_${partIndex}`, index), type: "text", text }
}
function reasoningPart(index: number, partIndex: number, length: number): MessagePart {
return {
id: id(`prt_reasoning_${partIndex}`, index),
type: "reasoning",
text: lorem(index * 19 + partIndex, length),
time: { start: 1700000000000 + index * 10_000, end: 1700000000000 + index * 10_000 + 500 },
}
}
function toolPart(
index: number,
partIndex: number,
tool: string,
input: Record<string, unknown>,
outputLength = 160,
): MessagePart {
const metadata =
tool === "apply_patch"
? { files: [patchFile(index, "update"), patchFile(index + 1, index % 2 === 0 ? "add" : "delete")] }
: tool === "edit" || tool === "write"
? {
filediff: fileDiff(String(input.filePath ?? `src/generated/file-${index}.ts`), index),
diff: patch(index, outputLength),
preview: patch(index + 1, 420),
}
: tool === "question"
? { answers: [["Proceed"], ["Keep sample output"]] }
: {}
return {
id: id(`prt_tool_${tool}_${partIndex}`, index),
type: "tool",
callID: id("call", index * 10 + partIndex),
tool,
state: {
status: "completed",
input,
output: lorem(index * 23 + partIndex, outputLength),
title: tool === "bash" ? "Verify generated output" : input.filePath || input.path || input.pattern || "completed",
metadata,
time: { start: 1700000000000 + index * 10_000, end: 1700000000000 + index * 10_000 + 400 },
},
}
}
function patchFile(seed: number, type: "add" | "update" | "delete") {
return {
filePath: `src/generated/patch-${seed}.ts`,
relativePath: `src/generated/patch-${seed}.ts`,
type,
additions: (seed % 7) + 1,
deletions: type === "add" ? 0 : seed % 4,
patch: patch(seed, 520),
before: type === "add" ? undefined : code(seed, 18),
after: type === "delete" ? undefined : code(seed + 1, 24),
}
}
function fileDiff(file: string, seed: number) {
const lines = seed % 12 === 0 ? 300 : seed % 8 === 0 ? 2 : 38
const before = code(seed, lines, seed % 10 === 0 ? 280 : 32)
const after =
lines === 2
? before.replace("value1", "updatedValue1")
: lines === 300
? code(seed + 1, lines, seed % 10 === 0 ? 280 : 32)
: before.replace("value4", "updatedValue4").replace("value20", "updatedValue20")
return {
file,
additions: lines === 300 ? 300 : lines === 2 ? 1 : 2,
deletions: lines === 300 ? 300 : lines === 2 ? 1 : 2,
before,
after,
}
}
function patch(seed: number, length: number) {
return `diff --git a/src/generated/file-${seed}.ts b/src/generated/file-${seed}.ts\n+${lorem(seed, length).replace(/\n/g, "\n+")}`
}
function code(seed: number, lines: number, width = 32) {
return Array.from(
{ length: lines },
(_, index) => `export const value${index} = "${lorem(seed + index, width)}"`,
).join("\n")
}
function turn(index: number): Message[] {
const diff = index % 9 === 0 ? [fileDiff(`src/generated/summary-${index}.ts`, index)] : []
const user = userMessage(targetID, index, 100 + (index % 4) * 80, diff)
const parts = [
...(index % 5 === 0 ? [reasoningPart(index, 0, 420)] : []),
...(index % 3 === 0
? [
toolPart(index, 0, "read", { filePath: `src/generated/file-${index}.ts`, offset: 0, limit: 80 }, 220),
toolPart(index, 5, "glob", { path: directory, pattern: `**/*sample-${index}*.ts` }, 140),
toolPart(index, 1, "grep", { path: directory, pattern: `sample-${index}`, include: "*.ts" }, 180),
toolPart(index, 6, "list", { path: `src/generated/${index}` }, 120),
]
: []),
textPart(index, 2, 160 + (index % 6) * 90),
...(index % 4 === 0 ? [toolPart(index, 3, "edit", { filePath: `src/generated/file-${index}.ts` }, 700)] : []),
...(index % 6 === 0
? [toolPart(index, 7, "write", { filePath: `src/generated/write-${index}.ts`, content: code(index, 28) }, 560)]
: []),
...(index % 8 === 0
? [toolPart(index, 8, "apply_patch", { files: [`src/generated/patch-${index}.ts`] }, 620)]
: []),
...(index % 7 === 0
? [toolPart(index, 4, "bash", { command: "bun typecheck", description: "Verify generated output" }, 620)]
: []),
...(index % 10 === 0 ? [toolPart(index, 9, "webfetch", { url: "https://example.com/docs/sample" }, 120)] : []),
...(index % 11 === 0 ? [toolPart(index, 10, "websearch", { query: "sample movement notes" }, 240)] : []),
...(index % 13 === 0
? [
toolPart(
index,
11,
"question",
{ questions: [{ question: "Use generated fixture?" }, { question: "Keep same row shape?" }] },
120,
),
]
: []),
...(index % 17 === 0
? [toolPart(index, 12, "task", { description: "Inspect generated fixture", subagent_type: "explore" }, 160)]
: []),
]
return [user, assistantMessage(targetID, index, user.info.id, parts)]
}
const targetMessages = Array.from({ length: 72 }, (_, index) => turn(index)).flat()
const sourceMessages = Array.from({ length: 12 }, (_, index) => [
userMessage(sourceID, index + 1000, 120),
assistantMessage(sourceID, index + 1000, id("msg_user", index + 1000), [textPart(index + 1000, 0, 240)]),
]).flat()
function renderable(part: MessagePart) {
if (part.type === "tool" && part.tool === "todowrite") return false
if (part.type === "text") return !!part.text.trim()
if (part.type === "reasoning") return !!part.text.trim()
return part.type !== "step-start" && part.type !== "step-finish" && part.type !== "patch"
}
function orderedParts(message: Message) {
return message.parts.slice().sort((a, b) => a.id.localeCompare(b.id))
}
export const fixture = {
directory,
project: {
id: projectID,
worktree: directory,
vcs: "git",
name: "smoke-project",
time: { created: 1700000000000, updated: 1700000000000 },
sandboxes: [],
},
provider: {
all: [
{
id: "opencode",
name: "OpenCode",
models: { "claude-opus-4-6": { id: "claude-opus-4-6", name: "Claude Opus 4.6", limit: { context: 200_000 } } },
},
],
connected: ["opencode"],
default: { providerID: "opencode", modelID: "claude-opus-4-6" },
},
sessions: [
{
id: sourceID,
slug: "source",
projectID,
directory,
title: "Uncommitted changes inquiry",
version: "dev",
time: { created: 1700000000000, updated: 1700000000000 },
},
{
id: targetID,
slug: "target",
projectID,
directory,
title: "Example Game: sample jump movement & sample physics analysis",
version: "dev",
time: { created: 1700000001000, updated: 1700000001000 },
},
],
sourceID,
targetID,
messages: { [sourceID]: sourceMessages, [targetID]: targetMessages },
expected: {
sourceTitle: "Uncommitted changes inquiry",
targetTitle: "Example Game: sample jump movement & sample physics analysis",
sourceMessageIDs: sourceMessages
.filter((message) => message.info.role === "user")
.map((message) => message.info.id),
targetMessageIDs: targetMessages
.filter((message) => message.info.role === "user")
.map((message) => message.info.id),
targetPartIDs: targetMessages.flatMap((message) =>
orderedParts(message)
.filter(renderable)
.map((part) => part.id),
),
},
}
export function pageMessages(sessionID: string, limit: number, before?: string) {
const messages = fixture.messages[sessionID as keyof typeof fixture.messages] ?? []
const end = before
? Math.max(
0,
messages.findIndex((message) => message.info.id === before),
)
: messages.length
const start = Math.max(0, end - limit)
return {
items: messages.slice(start, end),
cursor: start > 0 ? messages[start]!.info.id : undefined,
}
}
@@ -0,0 +1,67 @@
import type { Page } from "@playwright/test"
import { base64Encode } from "@opencode-ai/core/util/encode"
import { mockOpenCodeServer } from "../../utils/mock-server"
import { fixture } from "./session-timeline-stress.fixture"
export async function installTimelineSettings(page: Page) {
await page.addInitScript(() => {
localStorage.setItem(
"settings.v3",
JSON.stringify({
general: {
editToolPartsExpanded: true,
shellToolPartsExpanded: true,
showReasoningSummaries: true,
showSessionProgressBar: true,
},
}),
)
})
}
export function mockStressTimeline(page: Page) {
return mockOpenCodeServer(page, {
sessions: fixture.sessions,
provider: fixture.provider,
directory: fixture.directory,
project: fixture.project,
pageMessages: (sessionID) => ({ items: fixture.messages[sessionID as keyof typeof fixture.messages] ?? [] }),
})
}
export async function installStressSessionTabs(page: Page) {
const server = `http://${process.env.PLAYWRIGHT_SERVER_HOST ?? "127.0.0.1"}:${process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"}`
await page.addInitScript(
({ directory, sourceID, targetID, dirBase64, server }) => {
localStorage.setItem(
"opencode.global.dat:server",
JSON.stringify({
projects: { local: [{ worktree: directory, expanded: true }] },
lastProject: { local: directory },
}),
)
localStorage.setItem(
"opencode.global.dat:tabs",
JSON.stringify(
[sourceID, targetID].map((sessionId) => ({
type: "session",
server,
dirBase64,
sessionId,
})),
),
)
},
{
directory: fixture.directory,
sourceID: fixture.sourceID,
targetID: fixture.targetID,
dirBase64: base64Encode(fixture.directory),
server,
},
)
}
export function stressSessionHref(sessionID: string) {
return `/${base64Encode(fixture.directory)}/session/${sessionID}`
}
@@ -0,0 +1,15 @@
import { expect, test } from "bun:test"
import { mkdtemp, rm } from "node:fs/promises"
import path from "node:path"
import os from "node:os"
import { prepareChromeTrace } from "../chrome-trace"
test("creates the configured trace directory", async () => {
const root = await mkdtemp(path.join(os.tmpdir(), "opencode-trace-"))
try {
const file = await prepareChromeTrace(path.join(root, "nested", "traces"), "session/tab", false, "test")
expect(file).toEndWith("-session-tab-458ed9e3-test.json")
} finally {
await rm(root, { recursive: true, force: true })
}
})
@@ -0,0 +1,42 @@
import { expect, test } from "bun:test"
import { compressCachedRepaintTrace, layoutShiftSample } from "../timeline/session-tab-repaint-probe"
test("compresses repeated repaint states without losing frame samples", () => {
const state = {
root: 1,
scrollTop: 10,
scrollHeight: 20,
bottomErrorPx: 0,
last: true,
rows: [{ key: "row", node: 2, top: 0, bottom: 10 }],
mounted: 1,
center: "content",
}
const trace = {
timeOriginEpochMs: 1_000,
startedAtPerformanceMs: 100,
samples: [
{ observedAtMs: 16, ...state, destination: ["target"], source: [] },
{ observedAtMs: 32, ...state, destination: ["target"], source: [] },
{ observedAtMs: 48, ...state, scrollTop: 11, destination: ["target"], source: [] },
],
mutations: [{ observedAtMs: 20, changed: [{ type: "add", node: 2 }] }],
shifts: [{ occurredAtMs: 24, value: 0.1 }],
windowMs: 1_000,
running: false,
stop() {},
}
const compressed = compressCachedRepaintTrace(trace)
const samples = compressed.samples.flatMap((group) =>
group.observedAtMs.map((observedAtMs) => ({ observedAtMs, ...group.state })),
)
expect(samples).toEqual(trace.samples)
expect(compressed.mutations).toEqual(trace.mutations)
expect(compressed.shifts).toEqual(trace.shifts)
})
test("records layout shifts at occurrence time within the probe window", () => {
expect(layoutShiftSample({ startTime: 99, value: 0.1 }, 100)).toBeUndefined()
expect(layoutShiftSample({ startTime: 124, value: 0.2 }, 100)).toEqual({ occurredAtMs: 24, value: 0.2 })
})
@@ -0,0 +1,54 @@
import { expect, test } from "bun:test"
import { classifySessionSwitch } from "../timeline/session-tab-switch-metrics"
test("counts source and blank samples before the destination is observed", () => {
const result = classifySessionSwitch([
{ observedAtMs: 16, destination: [], source: ["source"], hasVisibleRows: true, last: false },
{ observedAtMs: 32, destination: [], source: [], hasVisibleRows: false, last: false },
{ observedAtMs: 48, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
{ observedAtMs: 64, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
{ observedAtMs: 80, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
])
expect(result.blankSamples).toBe(1)
expect(result.sourceSamples).toBe(1)
expect(result.unknownSamples).toBe(0)
expect(result.firstDestinationObservedMs).toBe(48)
expect(result.stableObservedMs).toBe(80)
})
test("does not classify mixed source and destination content as correct", () => {
const result = classifySessionSwitch([
{
observedAtMs: 16,
destination: ["destination"],
source: ["source"],
hasVisibleRows: true,
last: true,
bottomErrorPx: 0,
},
{ observedAtMs: 32, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
{ observedAtMs: 48, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
{ observedAtMs: 64, destination: ["destination"], source: [], hasVisibleRows: true, last: true, bottomErrorPx: 0 },
])
expect(result.firstCorrectObservedMs).toBe(32)
expect(result.stableObservedMs).toBe(64)
})
test("reports missing correctness without throwing", () => {
const result = classifySessionSwitch([
{
observedAtMs: 16,
destination: ["destination"],
source: ["source"],
hasVisibleRows: true,
last: true,
bottomErrorPx: 0,
},
])
expect(result.firstDestinationObservedMs).toBe(16)
expect(result.firstCorrectObservedMs).toBeNull()
expect(result.stableObservedMs).toBeNull()
})
@@ -0,0 +1,14 @@
import { expect, test } from "bun:test"
import { streamChunk } from "../timeline/session-timeline-benchmark.fixture"
import { streamProgress } from "../timeline/session-timeline-stream-probe"
test("classifies emitted stream markers using the fixture cycle", () => {
expect(streamProgress("before stream-17 after stream-18")).toEqual({ index: 18, phase: "boundary" })
expect(streamProgress("before stream-18 after stream-19")).toEqual({ index: 19, phase: "stream" })
expect(streamProgress("benchmark-complete stream-36")).toEqual({ index: 36, phase: "complete" })
expect(streamProgress("no marker")).toEqual({ index: -1, phase: "unknown" })
})
test("emits progress markers at fixture boundaries", () => {
expect(streamProgress(streamChunk(18, 160))).toEqual({ index: 18, phase: "boundary" })
})
@@ -0,0 +1,16 @@
import { expect, test } from "bun:test"
import { layoutShiftValue, removeVisibleRow } from "../timeline/session-timeline-stream-probe"
test("excludes layout shifts before the probe window and recent input", () => {
expect(layoutShiftValue({ startTime: 9, value: 0.1 }, 10)).toBeUndefined()
expect(layoutShiftValue({ startTime: 10, value: 0.2, hadRecentInput: true }, 10)).toBeUndefined()
expect(layoutShiftValue({ startTime: 11, value: 0.3 }, 10)).toBe(0.3)
})
test("classifies removed rows from their last painted visibility", () => {
const row = {}
const visible = new Set([row])
expect(removeVisibleRow(visible, row)).toBe(true)
expect(removeVisibleRow(visible, row)).toBe(false)
})
+6 -2
View File
@@ -44,7 +44,10 @@ export async function mockOpenCodeServer(page: Page, config: MockServerConfig) {
await page.route("**/*", async (route) => {
const url = new URL(route.request().url())
const targetPort = process.env.PLAYWRIGHT_SERVER_PORT ?? "4096"
if (url.port !== targetPort) return route.fallback()
const appPort = new URL(
process.env.PLAYWRIGHT_BASE_URL ?? `http://127.0.0.1:${process.env.PLAYWRIGHT_PORT ?? "3000"}`,
).port
if (url.port !== targetPort && url.port !== appPort) return route.fallback()
const path = url.pathname
if (path === "/global/event" || path === "/event") return sse(route, config.events?.(), config.eventRetry)
@@ -72,7 +75,8 @@ export async function mockOpenCodeServer(page: Page, config: MockServerConfig) {
return json(route, pageData.items, pageData.cursor ? { "x-next-cursor": pageData.cursor } : undefined)
}
return json(route, {})
if (url.port === targetPort && targetPort !== appPort) return json(route, {})
return route.fallback()
})
}
+2 -1
View File
@@ -24,7 +24,8 @@
"test:e2e": "playwright test",
"test:e2e:local": "playwright test",
"test:e2e:ui": "playwright test --ui",
"test:e2e:report": "playwright show-report e2e/playwright-report"
"test:e2e:report": "playwright show-report e2e/playwright-report",
"test:bench": "bun test ./e2e/performance/unit && playwright test --config e2e/performance/playwright.config.ts"
},
"license": "MIT",
"devDependencies": {
+1
View File
@@ -9,6 +9,7 @@ const reuse = !process.env.CI
const workers = Number(process.env.PLAYWRIGHT_WORKERS ?? (process.env.CI ? 5 : 0)) || undefined
export default defineConfig({
testDir: "./e2e",
testIgnore: process.env.OPENCODE_PERFORMANCE === "1" ? "performance/**/*.test.ts" : "performance/**",
outputDir: "./e2e/test-results",
timeout: 60_000,
expect: {
+1 -1
View File
@@ -23,7 +23,7 @@ export const Plugin = PluginV2.define({
skill: new SkillV2.Info({
name: "customize-opencode",
description:
"Use ONLY when the user is editing or creating opencode's own configuration: opencode.json, opencode.jsonc, files under .opencode/, or files under ~/.config/opencode/. Also use when creating or fixing opencode agents, subagents, skills, plugins, MCP servers, or permission rules. Do not use for the user's own application code, or for any project that is not configuring opencode itself.",
"Use ONLY when the user is editing or creating opencode's own configuration: opencode.json, opencode.jsonc, files under .opencode/, or files under ~/.config/opencode/. Also use when creating or fixing opencode agents, subagents, commands, skills, plugins, MCP servers, or permission rules. Do not use for the user's own application code, or for any project that is not configuring opencode itself.",
location: AbsolutePath.make("/builtin/customize-opencode.md"),
content: CustomizeOpencodeContent,
}),
@@ -43,6 +43,8 @@ already-loaded config until then.
| Global config | `~/.config/opencode/opencode.json` (NOT `~/.opencode/`) |
| Project agents | `.opencode/agent/<name>.md` or `.opencode/agents/<name>.md` |
| Global agents | `~/.config/opencode/agent(s)/<name>.md` |
| Project commands | `.opencode/command/<name>.md` or `.opencode/commands/<name>.md` |
| Global commands | `~/.config/opencode/command(s)/<name>.md` |
| Project skills | `.opencode/skill(s)/<name>/SKILL.md` |
| Global skills | `~/.config/opencode/skill(s)/<name>/SKILL.md` |
| External skills (auto-loaded) | `~/.claude/skills/<name>/SKILL.md`, `~/.agents/skills/<name>/SKILL.md` |
@@ -96,7 +98,7 @@ Every field is optional.
},
"command": {
"deploy": { "description": "...", "prompt": "..." }
"deploy": { "description": "...", "template": "..." }
},
"provider": {
@@ -151,6 +153,7 @@ Shape notes worth being explicit about:
- `skills` is an object with `paths` and/or `urls`, not an array.
- `references` is an object keyed by alias. Each value is a local path, Git repository, or string shorthand.
- `agent` is an object keyed by agent name, not an array.
- `command` is an object keyed by command name, not an array.
- `plugin` is an array of strings or `[name, options]` tuples, not an object.
- `mcp[name].command` is an array of strings, never a single string. `type` is required.
- `permission` is either a string action or an object keyed by tool name.
@@ -277,6 +280,31 @@ opencode ships with `build`, `plan`, `general`, `explore`. Hidden internal agent
`compaction`, `title`, `summary`. To override a built-in's fields, define the
same key in `agent: { <name>: { ... } }`.
## Commands
opencode's command loader scans for `**/*.md` inside command directories. The
file is named after the command, and lives directly inside the `command` folder:
```
.opencode/command/deploy.md
```
Frontmatter:
```markdown
---
description: One sentence describing what the command does.
agent: build
model: anthropic/claude-sonnet-4-6
---
(command body in markdown: the prompt opencode runs, with $ARGUMENTS for the user's input)
```
- `template` is the command body — everything below the frontmatter — and is required: it is the prompt opencode runs when the command is invoked. Do not also put a `template:` key in the frontmatter.
- `$ARGUMENTS` is replaced with everything the user typed after the command; `$1`, `$2`, … pull individual positional arguments.
- Optional: `description`, `agent`, `model`, `variant`, `subtask`.
## Plugins
`plugin:` is an array. Each entry is one of:
@@ -415,8 +443,8 @@ When a user's config is broken and opencode won't start, these env vars help:
exact shape, or the field is not covered in this skill, fetch
`https://opencode.ai/config.json` and read the schema rather than guessing.
- Preserve `$schema` and any existing fields the user did not ask to change.
- For agent, skill, and plugin definitions, prefer creating new files in the
correct location over inlining everything in `opencode.json`.
- For agent, command, skill, and plugin definitions, prefer creating new files
in the correct location over inlining everything in `opencode.json`.
- If the user's existing config is malformed, point them at the env-var escape
hatches above so they can edit from inside opencode without breaking their
session.
+1 -10
View File
@@ -1,7 +1,7 @@
export * as SessionRunner from "./index"
import type { LLMError } from "@opencode-ai/llm"
import { Context, Effect, Schema } from "effect"
import { Context, Effect } from "effect"
import { SessionSchema } from "../schema"
import type { ContextSnapshotDecodeError, MessageDecodeError } from "../error"
import { SessionRunnerModel } from "./model"
@@ -9,20 +9,11 @@ import type { SystemContext } from "../../system-context/index"
import type { SessionContextEpoch } from "../context-epoch"
import type { ToolOutputStore } from "../../tool-output-store"
export class StepLimitExceededError extends Schema.TaggedErrorClass<StepLimitExceededError>()(
"SessionRunner.StepLimitExceededError",
{
sessionID: SessionSchema.ID,
limit: Schema.Int,
},
) {}
export type RunError =
| LLMError
| SessionRunnerModel.Error
| MessageDecodeError
| ContextSnapshotDecodeError
| StepLimitExceededError
| SystemContext.InitializationBlocked
| SessionContextEpoch.AgentReplacementBlocked
| ToolOutputStore.Error
+26 -22
View File
@@ -3,6 +3,7 @@ import {
LLMClient,
LLMError,
LLMEvent,
Message,
SystemPart,
isContextOverflowFailure,
type ProviderErrorEvent,
@@ -29,10 +30,11 @@ import { SessionHistory } from "../history"
import { SessionInput } from "../input"
import { SessionSchema } from "../schema"
import { SessionStore } from "../store"
import { type RunError, Service, StepLimitExceededError } from "./index"
import { type RunError, Service } from "./index"
import { SessionRunnerModel } from "./model"
import { createLLMEventPublisher } from "./publish-llm-event"
import { toLLMMessages } from "./to-llm-message"
import { MAX_STEPS_PROMPT } from "./max-steps"
/**
* Runs one durable coding-agent Session until it settles.
@@ -45,7 +47,7 @@ import { toLLMMessages } from "./to-llm-message"
* - [ ] Replace local ownership with durable multi-node ownership when clustered.
* - [ ] Mark busy, retrying, idle, interrupted, or terminal-failure status durably.
* - [ ] Honor interruption and reject stale work after runtime attachment replacement.
* - [x] Bound model steps.
* - [x] Honor optional agent step limits.
* - [ ] Bound provider retries and repeated identical tool calls.
*
* - Runtime context assembly
@@ -80,13 +82,10 @@ import { toLLMMessages } from "./to-llm-message"
* Durable activity recovery remains a separate future slice with an explicit retry policy.
*
* The current slice loads V2 history, translates it, resolves a model through a core service, and persists one
* provider turn. Registry definitions are advertised, local tool calls are settled durably, and a
* bounded explicit loop starts the next provider turn after local settlement.
* provider turn. Registry definitions are advertised, local tool calls are settled durably, and an
* explicit loop starts the next provider turn after local settlement. Configured agent step limits bound the loop.
*/
// QUESTION: Did this exist previously, or did we add this limit? Does it make sense?
const MAX_STEPS = 25
export const layer = Layer.effect(
Service,
Effect.gen(function* () {
@@ -175,6 +174,7 @@ export const layer = Layer.effect(
const runTurnAttempt = Effect.fn("SessionRunner.runTurn")(function* (
sessionID: SessionSchema.ID,
promotion: SessionInput.Delivery | undefined,
step: number,
recoverOverflow?: typeof compaction.compactAfterOverflow,
) {
const session = yield* getSession(sessionID)
@@ -214,7 +214,8 @@ export const layer = Layer.effect(
const model = yield* models.resolve(session)
const entries = yield* SessionHistory.entriesForRunner(db, session.id, system.baselineSeq)
const context = entries.map((entry) => entry.message)
const toolMaterialization = yield* tools.materialize(agent.info?.permissions)
const isLastStep = agent.info?.steps !== undefined && step >= agent.info.steps
const toolMaterialization = isLastStep ? undefined : yield* tools.materialize(agent.info?.permissions)
const promptCacheKey = /^ses_[0-9a-f]{64}$/.test(session.id) ? session.id.slice(4) : session.id
const request = LLM.request({
model,
@@ -222,8 +223,9 @@ export const layer = Layer.effect(
system: [agent.info?.system, system.baseline]
.filter((part): part is string => part !== undefined && part.length > 0)
.map(SystemPart.make),
messages: toLLMMessages(context, model),
tools: toolMaterialization.definitions,
messages: [...toLLMMessages(context, model), ...(isLastStep ? [Message.assistant(MAX_STEPS_PROMPT)] : [])],
tools: toolMaterialization?.definitions ?? [],
toolChoice: isLastStep ? "none" : undefined,
})
if (yield* compaction.compactIfNeeded({ sessionID: session.id, entries, model, request }))
return yield* Effect.die(rebuildPreparedTurn())
@@ -254,6 +256,10 @@ export const layer = Layer.effect(
}
yield* publish(event)
if (event.type !== "tool-call" || event.providerExecuted) return
if (!toolMaterialization) {
yield* withPublication(publisher.failUnsettledTools("Tools are disabled after the maximum agent steps"))
return
}
needsContinuation = true
const assistantMessageID = yield* publisher.assistantMessageID(event.id)
yield* Effect.uninterruptibleMask((restore) =>
@@ -340,31 +346,32 @@ export const layer = Layer.effect(
type RunTurn = (
sessionID: SessionSchema.ID,
promotion: SessionInput.Delivery | undefined,
step: number,
) => Effect.Effect<boolean, RunError>
const runAfterOverflowCompaction: RunTurn = Effect.fnUntraced(function* (sessionID, promotion) {
return yield* runTurnAttempt(sessionID, promotion).pipe(
const runAfterOverflowCompaction: RunTurn = Effect.fnUntraced(function* (sessionID, promotion, step) {
return yield* runTurnAttempt(sessionID, promotion, step).pipe(
Effect.catchDefect(
Effect.fnUntraced(function* (defect) {
if (!(defect instanceof TurnTransitionError)) return yield* Effect.die(defect)
if (defect.transition._tag === "ContinueAfterOverflowCompaction")
return yield* Effect.die("Post-compaction provider attempt cannot recover another overflow")
yield* Effect.yieldNow
return yield* runAfterOverflowCompaction(sessionID, defect.transition.promotion)
return yield* runAfterOverflowCompaction(sessionID, defect.transition.promotion, step)
}),
),
)
})
const runTurn: RunTurn = Effect.fnUntraced(function* (sessionID, promotion) {
return yield* runTurnAttempt(sessionID, promotion, compaction.compactAfterOverflow).pipe(
const runTurn: RunTurn = Effect.fnUntraced(function* (sessionID, promotion, step) {
return yield* runTurnAttempt(sessionID, promotion, step, compaction.compactAfterOverflow).pipe(
Effect.catchDefect(
Effect.fnUntraced(function* (defect) {
if (!(defect instanceof TurnTransitionError)) return yield* Effect.die(defect)
yield* Effect.yieldNow
if (defect.transition._tag === "ContinueAfterOverflowCompaction")
return yield* runAfterOverflowCompaction(sessionID, undefined)
return yield* runTurn(sessionID, defect.transition.promotion)
return yield* runAfterOverflowCompaction(sessionID, undefined, step)
return yield* runTurn(sessionID, defect.transition.promotion, step)
}),
),
)
@@ -382,14 +389,11 @@ export const layer = Layer.effect(
let openActivity = input.force === true || hasSteer || hasQueue
while (openActivity) {
let needsContinuation = true
for (let step = 0; step < MAX_STEPS; step++) {
needsContinuation = yield* runTurn(input.sessionID, promotion)
for (let step = 1; needsContinuation; step++) {
needsContinuation = yield* runTurn(input.sessionID, promotion, step)
promotion = "steer"
if (!needsContinuation) needsContinuation = yield* SessionInput.hasPending(db, input.sessionID, "steer")
if (!needsContinuation) break
}
if (needsContinuation)
return yield* new StepLimitExceededError({ sessionID: input.sessionID, limit: MAX_STEPS })
openActivity = yield* SessionInput.hasPending(db, input.sessionID, "queue")
promotion = openActivity ? "queue" : undefined
}
@@ -1,4 +1,4 @@
CRITICAL - MAXIMUM STEPS REACHED
export const MAX_STEPS_PROMPT = `CRITICAL - MAXIMUM STEPS REACHED
The maximum number of steps allowed for this task has been reached. Tools are disabled until next user input. Respond with text only.
@@ -13,4 +13,4 @@ Response must include:
- List of any remaining tasks that were not completed
- Recommendations for what should be done next
Any attempt to use tools is a critical violation. Respond with text ONLY.
Any attempt to use tools is a critical violation. Respond with text ONLY.`
+54 -56
View File
@@ -1461,6 +1461,7 @@ describe("SessionRunnerLLM", () => {
})
requests.length = 0
executions.length = 0
responses = [
fragmentFixture("text", "text-summary-2", ["## Goal\n- Preserve the updated task"]).completeEvents,
fragmentFixture("text", "text-final-2", ["Continued again"]).completeEvents,
@@ -3177,7 +3178,7 @@ describe("SessionRunnerLLM", () => {
}),
)
it.effect("fails after the bounded number of local tool continuation steps", () =>
it.effect("continues past 25 local tool steps when the agent has no step limit", () =>
Effect.gen(function* () {
yield* setup
const session = yield* SessionV2.Service
@@ -3188,62 +3189,10 @@ describe("SessionRunnerLLM", () => {
executions.length = 0
streamGate = undefined
streamStarted = undefined
responses = Array.from({ length: 25 }, (_, index) => [
LLMEvent.stepStart({ index: 0 }),
LLMEvent.toolCall({ id: `call-echo-${index}`, name: "echo", input: { text: `${index}` } }),
LLMEvent.stepFinish({ index: 0, reason: "tool-calls" }),
LLMEvent.finish({ reason: "tool-calls" }),
])
const failure = yield* session.resume(sessionID).pipe(Effect.flip)
expect(failure).toMatchObject({ _tag: "SessionRunner.StepLimitExceededError", sessionID, limit: 25 })
expect(requests).toHaveLength(25)
expect(executions).toHaveLength(25)
}),
)
it.effect("does not restart a capped tool loop for a coalesced stale wake", () =>
Effect.gen(function* () {
yield* setup
const session = yield* SessionV2.Service
const coordinator = yield* SessionRunCoordinator.Service
yield* session.prompt({ sessionID, prompt: new Prompt({ text: "Loop forever" }), resume: false })
requests.length = 0
responses = Array.from({ length: 25 }, (_, index) => [
LLMEvent.stepStart({ index: 0 }),
LLMEvent.toolCall({ id: `call-capped-${index}`, name: "echo", input: { text: `${index}` } }),
LLMEvent.stepFinish({ index: 0, reason: "tool-calls" }),
LLMEvent.finish({ reason: "tool-calls" }),
])
streamGate = yield* Deferred.make<void>()
streamStarted = yield* Deferred.make<void>()
const run = yield* session.resume(sessionID).pipe(Effect.forkChild)
yield* Deferred.await(streamStarted)
yield* coordinator.wake(sessionID)
yield* Deferred.succeed(streamGate, undefined)
expect(yield* Fiber.join(run).pipe(Effect.flip)).toMatchObject({ _tag: "SessionRunner.StepLimitExceededError" })
streamGate = undefined
streamStarted = undefined
yield* Effect.yieldNow
expect(requests).toHaveLength(25)
}),
)
it.effect("accepts a terminal response on the final bounded provider turn", () =>
Effect.gen(function* () {
yield* setup
const session = yield* SessionV2.Service
yield* session.prompt({ sessionID, prompt: new Prompt({ text: "Finish at the limit" }), resume: false })
requests.length = 0
responses = [
...Array.from({ length: 24 }, (_, index) => [
...Array.from({ length: 25 }, (_, index) => [
LLMEvent.stepStart({ index: 0 }),
LLMEvent.toolCall({ id: `call-terminal-${index}`, name: "echo", input: { text: `${index}` } }),
LLMEvent.toolCall({ id: `call-echo-${index}`, name: "echo", input: { text: `${index}` } }),
LLMEvent.stepFinish({ index: 0, reason: "tool-calls" }),
LLMEvent.finish({ reason: "tool-calls" }),
]),
@@ -3256,7 +3205,56 @@ describe("SessionRunnerLLM", () => {
yield* session.resume(sessionID)
expect(requests).toHaveLength(25)
expect(requests).toHaveLength(26)
expect(executions).toHaveLength(25)
}),
)
it.effect("forces a text response on an agent's configured final step", () =>
Effect.gen(function* () {
yield* setup
const agents = yield* AgentV2.Service
yield* agents.update((editor) =>
editor.update(AgentV2.ID.make("build"), (agent) => {
agent.steps = 2
}),
)
const session = yield* SessionV2.Service
yield* session.prompt({ sessionID, prompt: new Prompt({ text: "Finish at the limit" }), resume: false })
requests.length = 0
executions.length = 0
responses = [
[
LLMEvent.stepStart({ index: 0 }),
LLMEvent.toolCall({ id: "call-terminal", name: "echo", input: { text: "done" } }),
LLMEvent.stepFinish({ index: 0, reason: "tool-calls" }),
LLMEvent.finish({ reason: "tool-calls" }),
],
[
LLMEvent.stepStart({ index: 0 }),
LLMEvent.toolCall({ id: "call-forbidden", name: "echo", input: { text: "forbidden" } }),
LLMEvent.stepFinish({ index: 0, reason: "tool-calls" }),
LLMEvent.finish({ reason: "tool-calls" }),
],
]
yield* session.resume(sessionID)
expect(requests).toHaveLength(2)
expect(requests[0]?.toolChoice).toBeUndefined()
expect(requests[1]?.toolChoice).toMatchObject({ type: "none" })
expect(requests[1]?.tools).toEqual([])
expect(requests[1]?.messages.at(-1)).toMatchObject({
role: "assistant",
content: [{ type: "text", text: expect.stringContaining("MAXIMUM STEPS REACHED") }],
})
expect(executions).toEqual(["done"])
expect(yield* session.context(sessionID)).toMatchObject([
{ type: "user", text: "Finish at the limit" },
{ type: "assistant", content: [{ type: "tool", id: "call-terminal", state: { status: "completed" } }] },
{ type: "assistant", content: [{ type: "tool", id: "call-forbidden", state: { status: "error" } }] },
])
}),
)
@@ -412,6 +412,9 @@ const fromRequest = Effect.fn("BedrockConverse.fromRequest")(function* (request:
stopSequences: generation?.stop,
},
toolConfig,
// Converse's base inferenceConfig has no topK; Anthropic/Nova accept it
// as a model-specific field, so it goes through additionalModelRequestFields.
additionalModelRequestFields: generation?.topK === undefined ? undefined : { top_k: generation.topK },
}
})
@@ -83,6 +83,26 @@ describe("Bedrock Converse route", () => {
}),
)
it.effect("passes topK through additionalModelRequestFields as top_k", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.prepare<BedrockConverse.BedrockConverseBody>(
LLM.updateRequest(baseRequest, { generation: { maxTokens: 64, temperature: 0, topK: 40 } }),
)
// Converse's inferenceConfig has no topK; Anthropic/Nova read it from
// additionalModelRequestFields as top_k.
expect(prepared.body.inferenceConfig).toEqual({ maxTokens: 64, temperature: 0 })
expect(prepared.body.additionalModelRequestFields).toEqual({ top_k: 40 })
}),
)
it.effect("omits additionalModelRequestFields when topK is unset", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.prepare<BedrockConverse.BedrockConverseBody>(baseRequest)
expect(prepared.body.additionalModelRequestFields).toBeUndefined()
}),
)
it.effect("lowers chronological system updates to wrapped user text in order", () =>
Effect.gen(function* () {
const prepared = yield* LLMClient.prepare<BedrockConverse.BedrockConverseBody>(
+24 -2
View File
@@ -218,7 +218,7 @@ function normalizeMessages(
if (
model.providerID === "mistral" ||
model.api.id.toLowerCase().includes("mistral") ||
model.api.id.toLocaleLowerCase().includes("devstral")
model.api.id.toLowerCase().includes("devstral")
) {
const scrub = (id: string) => {
return id
@@ -666,6 +666,9 @@ export function variants(model: Provider.Model): Record<string, Record<string, a
if (!model.capabilities.reasoning) return {}
const id = model.id.toLowerCase()
const glm52 = ["glm-5.2", "glm-5-2", "glm-5p2"].some(
(name) => id.includes(name) || model.api.id.toLowerCase().includes(name),
)
if (
model.api.id.toLowerCase().includes("minimax-m3") &&
["@ai-sdk/anthropic", "@ai-sdk/openai-compatible"].includes(model.api.npm)
@@ -677,13 +680,32 @@ export function variants(model: Provider.Model): Record<string, Record<string, a
}
const adaptiveThinkingOmitted = anthropicOmitsThinking(model.api.id)
const adaptiveEfforts = anthropicAdaptiveEfforts(model.api.id)
if (glm52 && model.api.npm === "@openrouter/ai-sdk-provider") {
// OpenRouter maps xhigh to GLM-5.2's native max effort.
return {
high: { reasoning: { effort: "high" } },
xhigh: { reasoning: { effort: "xhigh" } },
}
}
if (glm52 && model.api.npm === "@ai-sdk/openai-compatible") {
return {
high: { reasoningEffort: "high" },
max: { reasoningEffort: "max" },
}
}
if (glm52 && model.api.npm === "@ai-sdk/anthropic") {
return {
high: { effort: "high" },
max: { effort: "max" },
}
}
if (
id.includes("deepseek-chat") ||
id.includes("deepseek-reasoner") ||
id.includes("deepseek-r1") ||
id.includes("deepseek-v3") ||
id.includes("minimax") ||
id.includes("glm") ||
(id.includes("glm") && !glm52) ||
id.includes("kimi") ||
id.includes("k2p") ||
id.includes("qwen") ||
+5 -20
View File
@@ -16,7 +16,7 @@ import { SessionCompaction } from "./compaction"
import { SystemPrompt } from "./system"
import { Instruction } from "./instruction"
import { Plugin } from "../plugin"
import MAX_STEPS from "../session/prompt/max-steps.txt"
import { MAX_STEPS_PROMPT } from "@opencode-ai/core/session/runner/max-steps"
import { ToolRegistry } from "@/tool/registry"
import { MCP } from "../mcp"
import { LSP } from "@/lsp/lsp"
@@ -1304,24 +1304,6 @@ export const layer = Layer.effect(
if (step === 1)
yield* summary.summarize({ sessionID, messageID: lastUser.id }).pipe(Effect.ignore, Effect.forkIn(scope))
if (step > 1 && lastFinished) {
for (const m of msgs) {
if (m.info.role !== "user" || m.info.id <= lastFinished.id) continue
for (const p of m.parts) {
if (p.type !== "text" || p.ignored || p.synthetic) continue
if (!p.text.trim()) continue
p.text = [
"<system-reminder>",
"The user sent the following message:",
p.text,
"",
"Please address this message and continue with your tasks.",
"</system-reminder>",
].join("\n")
}
}
}
yield* plugin.trigger("experimental.chat.messages.transform", {}, { messages: msgs })
const [skills, env, instructions, modelMsgs] = yield* Effect.all([
@@ -1340,7 +1322,10 @@ export const layer = Layer.effect(
sessionID,
parentSessionID: session.parentID,
system,
messages: [...modelMsgs, ...(isLastStep ? [{ role: "assistant" as const, content: MAX_STEPS }] : [])],
messages: [
...modelMsgs,
...(isLastStep ? [{ role: "assistant" as const, content: MAX_STEPS_PROMPT }] : []),
],
tools,
model,
toolChoice: format.type === "json_schema" ? "required" : undefined,
@@ -2895,6 +2895,102 @@ describe("ProviderTransform.variants", () => {
expect(result).toEqual({})
})
test("glm-5.2 returns native effort variants for openai-compatible providers", () => {
const model = createMockModel({
id: "zhipuai/glm-5.2",
providerID: "zhipuai",
api: {
id: "glm-5.2",
url: "https://open.bigmodel.cn/api/paas/v4",
npm: "@ai-sdk/openai-compatible",
},
})
expect(ProviderTransform.variants(model)).toEqual({
high: { reasoningEffort: "high" },
max: { reasoningEffort: "max" },
})
})
test("recognizes GLM-5.2 provider model IDs", () => {
for (const id of ["accounts/fireworks/models/glm-5p2", "zai-org-glm-5-2", "umans-glm-5.2"]) {
const model = createMockModel({
id: `test/${id}`,
api: {
id,
url: "https://api.test.com",
npm: "@ai-sdk/openai-compatible",
},
})
expect(ProviderTransform.variants(model)).toEqual({
high: { reasoningEffort: "high" },
max: { reasoningEffort: "max" },
})
}
})
test("recognizes GLM-5.2 from the API ID when the configured model ID is an alias", () => {
const model = createMockModel({
id: "custom/my-glm",
api: {
id: "accounts/fireworks/models/glm-5p2",
url: "https://api.fireworks.ai/inference/v1",
npm: "@ai-sdk/openai-compatible",
},
})
expect(ProviderTransform.variants(model)).toEqual({
high: { reasoningEffort: "high" },
max: { reasoningEffort: "max" },
})
})
test("glm-5.2 returns openrouter effort variants for openrouter", () => {
const model = createMockModel({
id: "openrouter/z-ai/glm-5.2",
providerID: "openrouter",
api: {
id: "z-ai/glm-5.2",
url: "https://openrouter.ai/api/v1",
npm: "@openrouter/ai-sdk-provider",
},
})
expect(ProviderTransform.variants(model)).toEqual({
high: { reasoning: { effort: "high" } },
xhigh: { reasoning: { effort: "xhigh" } },
})
})
test("glm-5.2 returns effort variants for anthropic-compatible providers", () => {
const model = createMockModel({
id: "zai-coding-plan/glm-5.2",
providerID: "zai-coding-plan",
api: {
id: "glm-5.2",
url: "https://api.z.ai/api/anthropic",
npm: "@ai-sdk/anthropic",
},
})
expect(ProviderTransform.variants(model)).toEqual({
high: { effort: "high" },
max: { effort: "max" },
})
})
test("glm-5.2 falls back to provider defaults for other packages", () => {
const model = createMockModel({
id: "test/glm-5.2",
api: {
id: "glm-5.2",
url: "https://api.test.com",
npm: "@ai-sdk/amazon-bedrock",
},
})
expect(ProviderTransform.variants(model)).toEqual({
low: { reasoningConfig: { type: "enabled", maxReasoningEffort: "low" } },
medium: { reasoningConfig: { type: "enabled", maxReasoningEffort: "medium" } },
high: { reasoningConfig: { type: "enabled", maxReasoningEffort: "high" } },
})
})
test("mistral models with reasoning support return variants", () => {
const model = createMockModel({
id: "mistral/mistral-small-latest",
@@ -1346,7 +1346,9 @@ it.instance(
const inputs = yield* llm.inputs
expect(inputs).toHaveLength(2)
expect(JSON.stringify(inputs.at(-1)?.messages)).toContain("second")
const messages = inputs.at(-1)?.messages
if (!Array.isArray(messages)) throw new Error("expected LLM messages")
expect(messages.at(-1)).toEqual({ role: "user", content: "second" })
}),
3_000,
)
@@ -2,6 +2,7 @@ import { PermissionV1 } from "@opencode-ai/core/v1/permission"
import { describe, expect } from "bun:test"
import path from "path"
import { Effect } from "effect"
import { LayerNode } from "@opencode-ai/core/effect/layer-node"
import { CrossSpawnSpawner } from "@opencode-ai/core/cross-spawn-spawner"
import type { Tool } from "@/tool/tool"
import { assertExternalDirectoryEffect } from "../../src/tool/external-directory"
@@ -11,7 +12,7 @@ import type { Permission } from "../../src/permission"
import { SessionID, MessageID } from "../../src/session/schema"
import { testEffect } from "../lib/effect"
const it = testEffect(CrossSpawnSpawner.defaultLayer)
const it = testEffect(LayerNode.buildLayer(CrossSpawnSpawner.node))
const baseCtx: Omit<Tool.Context, "ask"> = {
sessionID: SessionID.make("ses_test"),
+5 -2
View File
@@ -7,8 +7,11 @@ import "./app.css"
function AppMeta() {
return (
<>
<Title>OpenCode Data</Title>
<Meta name="description" content="OpenCode usage data, market share, token cost, and session cost." />
<Title>AI Model Usage Rankings | OpenCode Data</Title>
<Meta
name="description"
content="Explore OpenCode Go usage across AI models, including token volume, rankings, market share, token pricing, session cost, cache ratio, and geo breakdowns."
/>
</>
)
}
+148 -94
View File
@@ -45,6 +45,7 @@ const statsUnfurlUrl = new URL(statsUnfurlPath, statsCanonicalBaseUrl).toString(
const modelHeaderLinks: readonly HeaderLink[] = [
{ href: "#overview", label: "Overview" },
{ href: "#usage", label: "Usage" },
{ href: "#users", label: "Users" },
{ href: "#efficiency", label: "Efficiency" },
{ href: "#geo-breakdown", label: "Geo Breakdown" },
{ href: "#peers", label: "Peers" },
@@ -120,11 +121,10 @@ export default function StatsModel() {
const [themePreference, setThemePreference] = createSignal<ThemePreference>("system")
const modelName = createMemo(() => catalogEntry()?.name ?? stats()?.model ?? modelParam() ?? "Model")
const labName = createMemo(() => formatCatalogLabName(catalogEntry()?.lab ?? stats()?.provider ?? labParam()))
const modelTitle = createMemo(() => `${modelName()} Data`)
const modelDescription = createMemo(() =>
stats()
? `${modelName()} usage, rank, token mix, cost, geo breakdown, and peer data across OpenCode Go.`
: `${modelName()} model facts, limits, and OpenCode Go usage availability.`,
const modelTitle = createMemo(() => `${modelName()} Usage, Cost & Rank | OpenCode Data`)
const modelDescription = createMemo(
() =>
`View ${modelName()} OpenCode Go usage data, including token volume, weekly rank, token mix, costs, cache ratio, sessions, geo breakdowns, and peer models.`,
)
const modelUrl = createMemo(() =>
new URL(
@@ -176,6 +176,7 @@ export default function StatsModel() {
<ModelHero data={stats() ?? null} catalog={catalogEntry() ?? null} labName={labName()} />
<ModelOverview data={stats() ?? null} />
<ModelUsageSection data={stats()?.usage ?? []} />
<ModelUsersSection data={stats()?.usage ?? []} />
<ModelEfficiencySection data={stats() ?? null} catalog={catalogEntry() ?? null} />
<ModelGeoBreakdownSection data={stats()?.country ?? emptyCountryRecord()} />
<ModelPeersSection data={stats() ?? null} />
@@ -359,14 +360,6 @@ function ModelOverview(props: { data: StatsModelData | null }) {
}
function ModelUsageSection(props: { data: ModelUsagePoint[] }) {
const [activeIndex, setActiveIndex] = createSignal<number>()
const max = createMemo(() => Math.max(0, ...props.data.map((item) => item.tokens)) || 1)
const activePoint = createMemo(() => {
const index = activeIndex()
if (index === undefined) return undefined
return props.data[index]
})
return (
<section id="usage" data-section="model-panel">
<SectionTitle title="Usage" description="Daily OpenCode Go token volume over the recent two-month window." />
@@ -374,92 +367,147 @@ function ModelUsageSection(props: { data: ModelUsagePoint[] }) {
when={props.data.some((item) => item.tokens > 0)}
fallback={<ModelEmptyState title="No usage" description="No usage landed in the current window." />}
>
<div
data-component="model-usage-chart"
data-dense-labels={isModelUsageDense(props.data.length) ? "true" : undefined}
role="img"
aria-label="Daily token usage chart"
style={{ "--model-usage-count": props.data.length } as JSX.CSSProperties}
onPointerLeave={(event) => {
if (event.pointerType === "touch") return
setActiveIndex(undefined)
}}
>
<div data-slot="model-usage-axis" aria-hidden="true">
<For each={props.data}>
{(point, index) => (
<div
data-active={activeIndex() === index() ? "true" : undefined}
data-label-hidden={isModelUsageLabelHidden(index(), props.data.length) ? "true" : undefined}
>
<span data-slot="model-usage-label">
<span data-slot="model-usage-total">{formatTokens(point.tokens)}</span>
<span data-slot="model-usage-date">{point.date}</span>
</span>
</div>
)}
</For>
</div>
<div data-slot="model-usage-bars">
<For each={props.data}>
{(point, index) => (
<div
data-slot="model-usage-column"
role="button"
tabIndex={0}
aria-label={`${point.date} ${formatTokens(point.tokens)} tokens`}
data-active={activeIndex() === index() ? "true" : undefined}
data-muted={activeIndex() !== undefined && activeIndex() !== index() ? "true" : undefined}
onPointerDown={(event) => {
if (event.pointerType !== "touch") return
setActiveIndex(index())
}}
onPointerEnter={() => setActiveIndex(index())}
onPointerMove={(event) => {
if (event.pointerType === "touch") return
setActiveIndex(index())
}}
onClick={() => setActiveIndex(index())}
onFocus={() => setActiveIndex(index())}
onBlur={() => setActiveIndex(undefined)}
onKeyDown={(event) => {
if (event.key !== "Enter" && event.key !== " ") return
event.preventDefault()
setActiveIndex(index())
}}
>
<div
data-slot="model-usage-bar"
style={{ "--model-usage-fill": `${modelUsageHeight(point.tokens, max())}%` } as JSX.CSSProperties}
/>
<Show when={activeIndex() === index() && activePoint()}>
{(active) => (
<div
data-component="chart-tooltip"
data-placement={index() > props.data.length * 0.62 ? "left" : "right"}
>
<strong>{active().date}</strong>
<span>{formatTokens(active().tokens)} tokens</span>
<div data-slot="tooltip-divider" />
<p>
<span data-slot="tooltip-label">
<i /> Daily tokens
</span>
<b>{formatTokens(active().tokens)}</b>
</p>
</div>
)}
</Show>
</div>
)}
</For>
</div>
</div>
<ModelColumnChart data={props.data} metric="tokens" ariaLabel="Daily token usage chart" />
</Show>
</section>
)
}
function ModelUsersSection(props: { data: ModelUsagePoint[] }) {
return (
<section id="users" data-section="model-panel">
<SectionTitle
title="Unique Users"
description="Daily unique OpenCode Go users over the recent two-month window."
/>
<Show
when={props.data.some((item) => item.users > 0)}
fallback={
<ModelEmptyState title="No user data" description="No user-bearing rows landed in the current window." />
}
>
<ModelColumnChart data={props.data} metric="users" ariaLabel="Daily unique user chart" />
</Show>
</section>
)
}
function ModelColumnChart(props: { data: ModelUsagePoint[]; metric: "tokens" | "users"; ariaLabel: string }) {
const [activeIndex, setActiveIndex] = createSignal<number>()
const max = createMemo(() => Math.max(0, ...props.data.map((item) => modelUsageMetricValue(item, props.metric))) || 1)
const activePoint = createMemo(() => {
const index = activeIndex()
if (index === undefined) return undefined
return props.data[index]
})
return (
<div
data-component="model-usage-chart"
data-metric={props.metric}
data-dense-labels={isModelUsageDense(props.data.length) ? "true" : undefined}
role="img"
aria-label={props.ariaLabel}
style={{ "--model-usage-count": props.data.length } as JSX.CSSProperties}
onPointerLeave={(event) => {
if (event.pointerType === "touch") return
setActiveIndex(undefined)
}}
>
<div data-slot="model-usage-axis" aria-hidden="true">
<For each={props.data}>
{(point, index) => (
<div
data-active={activeIndex() === index() ? "true" : undefined}
data-label-hidden={isModelUsageLabelHidden(index(), props.data.length) ? "true" : undefined}
>
<span data-slot="model-usage-label">
<span data-slot="model-usage-total">{formatModelUsageValue(point, props.metric)}</span>
<span data-slot="model-usage-date">{point.date}</span>
</span>
</div>
)}
</For>
</div>
<div data-slot="model-usage-bars">
<For each={props.data}>
{(point, index) => (
<div
data-slot="model-usage-column"
role="button"
tabIndex={0}
aria-label={`${point.date} ${formatModelUsageValue(point, props.metric)} ${modelUsageLabel(props.metric)}`}
data-active={activeIndex() === index() ? "true" : undefined}
data-muted={activeIndex() !== undefined && activeIndex() !== index() ? "true" : undefined}
onPointerDown={(event) => {
if (event.pointerType !== "touch") return
setActiveIndex(index())
}}
onPointerEnter={() => setActiveIndex(index())}
onPointerMove={(event) => {
if (event.pointerType === "touch") return
setActiveIndex(index())
}}
onClick={() => setActiveIndex(index())}
onFocus={() => setActiveIndex(index())}
onBlur={() => setActiveIndex(undefined)}
onKeyDown={(event) => {
if (event.key !== "Enter" && event.key !== " ") return
event.preventDefault()
setActiveIndex(index())
}}
>
<div
data-slot="model-usage-bar"
style={
{
"--model-usage-fill": `${modelUsageHeight(modelUsageMetricValue(point, props.metric), max())}%`,
} as JSX.CSSProperties
}
/>
<Show when={activeIndex() === index() && activePoint()}>
{(active) => (
<div
data-component="chart-tooltip"
data-placement={index() > props.data.length * 0.62 ? "left" : "right"}
>
<strong>{active().date}</strong>
<span>
{formatModelUsageValue(active(), props.metric)} {modelUsageLabel(props.metric)}
</span>
<div data-slot="tooltip-divider" />
<p>
<span data-slot="tooltip-label">
<i /> Daily {modelUsageLabel(props.metric)}
</span>
<b>{formatModelUsageValue(active(), props.metric)}</b>
</p>
</div>
)}
</Show>
</div>
)}
</For>
</div>
</div>
)
}
function modelUsageMetricValue(point: ModelUsagePoint, metric: "tokens" | "users") {
if (metric === "users") return point.users
return point.tokens
}
function formatModelUsageValue(point: ModelUsagePoint, metric: "tokens" | "users") {
if (metric === "users") return formatUsers(point.users)
return formatTokens(point.tokens)
}
function modelUsageLabel(metric: "tokens" | "users") {
if (metric === "users") return "users"
return "tokens"
}
function ModelEfficiencySection(props: { data: StatsModelData | null; catalog: ModelCatalogEntry | null }) {
return (
<section id="efficiency" data-section="model-panel">
@@ -835,6 +883,12 @@ function formatInteger(value: number) {
return new Intl.NumberFormat("en").format(value)
}
function formatUsers(value: number) {
if (value >= 1_000_000) return `${trimNumber(value / 1_000_000, value >= 10_000_000 ? 0 : 1)}M`
if (value >= 1_000) return `${trimNumber(value / 1_000, value >= 10_000 ? 0 : 1)}K`
return formatInteger(Math.round(value))
}
function formatPercent(value: number) {
return `${value.toFixed(value > 0 && value < 10 ? 1 : 0)}%`
}
@@ -69,10 +69,10 @@ export default function StatsLab() {
const githubStars = createAsync(() => getGitHubStars())
const [themePreference, setThemePreference] = createSignal<ThemePreference>("system")
const labName = createMemo(() => lab()?.name ?? formatCatalogLabName(labParam()))
const labTitle = createMemo(() => `${labName()} Models`)
const labTitle = createMemo(() => `${labName()} AI Model Usage & Rankings | OpenCode Data`)
const labDescription = createMemo(
() =>
`Explore ${labName()} models used in OpenCode, with recent token usage, context windows, release dates, and model-specific data.`,
`Compare ${labName()} models used in OpenCode Go, including token usage, model rankings, context windows, release dates, costs, and model-specific data.`,
)
const labUrl = createMemo(() => new URL(lab()?.id ?? labParam(), statsCanonicalBaseUrl).toString())
const updateThemePreference = (preference: ThemePreference) => {
+63 -21
View File
@@ -1546,7 +1546,7 @@
color: var(--stats-text);
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] {
[data-page="stats"] :is([data-section="top-models"], [data-section="unique-users"]) [data-component="chart-tooltip"] {
top: 110px;
box-sizing: border-box;
display: flex;
@@ -1564,40 +1564,59 @@
color: var(--stats-text);
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"][data-placement="right"] {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"][data-placement="right"] {
right: auto;
left: calc(100% + 8px);
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"][data-placement="left"] {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"][data-placement="left"] {
right: calc(100% + 8px);
left: auto;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] strong,
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] > span {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
strong,
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
> span {
display: block;
font-size: 11px;
line-height: 12px;
white-space: nowrap;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] strong {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
strong {
padding: 8px 8px 0;
font-weight: 500;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] > span {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
> span {
padding: 4px 8px 8px;
color: var(--stats-muted);
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] [data-slot="tooltip-divider"] {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
[data-slot="tooltip-divider"] {
height: 0.5px;
margin: 0;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] p {
[data-page="stats"] :is([data-section="top-models"], [data-section="unique-users"]) [data-component="chart-tooltip"] p {
grid-template-columns: minmax(0, 1fr) auto;
gap: 4px;
height: 16px;
@@ -1608,36 +1627,50 @@
line-height: 12px;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] p[data-muted="true"] {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
p[data-muted="true"] {
opacity: 0.46;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] [data-slot="tooltip-divider"] + p {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
[data-slot="tooltip-divider"]
+ p {
margin-top: 8px;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] p:last-child {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
p:last-child {
margin-bottom: 8px;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] [data-slot="tooltip-label"] {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
[data-slot="tooltip-label"] {
grid-template-columns: 16px minmax(0, 1fr);
gap: 4px;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] i {
[data-page="stats"] :is([data-section="top-models"], [data-section="unique-users"]) [data-component="chart-tooltip"] i {
width: 6px;
height: 6px;
justify-self: center;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] b {
[data-page="stats"] :is([data-section="top-models"], [data-section="unique-users"]) [data-component="chart-tooltip"] b {
font-weight: 500;
}
[data-page="stats"]
:is(
[data-section="leaderboard"],
[data-section="unique-users"],
[data-section="market-share"],
[data-section="geo-breakdown"],
[data-section="token-cost"],
@@ -3264,10 +3297,12 @@
background: #242424f2;
}
[data-page="stats"][data-theme="dark"] [data-section="top-models"] [data-component="chart-tooltip"],
[data-page="stats"][data-theme="dark"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"],
:root[data-stats-theme="dark"]
[data-page="stats"]:not([data-theme="light"])
[data-section="top-models"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"] {
background: #242424f2;
box-shadow:
@@ -3276,10 +3311,13 @@
0 4px 8px #00000052;
}
[data-page="stats"][data-theme="dark"] [data-section="top-models"] [data-component="chart-tooltip"] > span,
[data-page="stats"][data-theme="dark"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
> span,
:root[data-stats-theme="dark"]
[data-page="stats"]:not([data-theme="light"])
[data-section="top-models"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"]
> span {
color: var(--stats-faint);
@@ -3544,6 +3582,7 @@
@media (max-width: 74rem) {
[data-page="stats"] [data-section="top-models"],
[data-page="stats"] [data-section="leaderboard"],
[data-page="stats"] [data-section="unique-users"],
[data-page="stats"] [data-section="market-share"],
[data-page="stats"] [data-section="geo-breakdown"],
[data-page="stats"] [data-section="token-cost"],
@@ -3703,6 +3742,7 @@
@media (max-width: 47.999rem) {
[data-page="stats"] [data-section="top-models"],
[data-page="stats"] [data-section="leaderboard"],
[data-page="stats"] [data-section="unique-users"],
[data-page="stats"] [data-section="market-share"],
[data-page="stats"] [data-section="geo-breakdown"],
[data-page="stats"] [data-section="token-cost"],
@@ -4011,7 +4051,7 @@
display: block;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"] {
[data-page="stats"] :is([data-section="top-models"], [data-section="unique-users"]) [data-component="chart-tooltip"] {
position: fixed;
top: auto;
right: 12px;
@@ -4025,7 +4065,9 @@
transform: none;
}
[data-page="stats"] [data-section="top-models"] [data-component="chart-tooltip"][data-placement] {
[data-page="stats"]
:is([data-section="top-models"], [data-section="unique-users"])
[data-component="chart-tooltip"][data-placement] {
right: 12px;
left: 12px;
}
+58 -7
View File
@@ -49,8 +49,9 @@ const rangeLabels: Record<UsageRange, string> = {
"1M": "1 Month",
"2M": "2 Months",
}
const statsHomeTitle = "OpenCode Data"
const statsHomeDescription = "OpenCode usage data, market share, token cost, and session cost."
const statsHomeTitle = "AI Model Usage Rankings | OpenCode Data"
const statsHomeDescription =
"Explore OpenCode Go usage across AI models, including token volume, rankings, market share, token pricing, session cost, cache ratio, and geo breakdowns."
const statsHomeFallbackUrl = "https://opencode.ai/data/"
const statsUnfurlPath = "banner.jpg"
const statsUnfurlAlt = "OpenCode Data wordmark on a dark patterned background"
@@ -170,6 +171,7 @@ export default function StatsHome() {
<>
<Hero updatedAt={stats().updatedAt} />
<TopModelsSection data={stats().usage} leaderboard={stats().leaderboard} />
<UniqueUsersSection data={stats().users} />
<SessionCostSection data={stats().sessionCost} />
<TokenCostSection data={stats().tokenCost} catalog={catalog() ?? null} />
<CacheRatioSection data={stats().cacheRatio} />
@@ -597,6 +599,8 @@ function FilterPills<T extends string>(props: {
function TopModelsChart(props: {
data: UsagePoint[]
range: UsageRange
metric?: "tokens" | "users"
ariaLabel?: string
activeModel: string | undefined
onActiveModelChange: (model: string | undefined) => void
}) {
@@ -605,6 +609,7 @@ function TopModelsChart(props: {
const maxTotal = createMemo(() => getTopModelsMaxTotal(props.data))
const segmentOrder = createMemo(() => getTopModelsSegmentOrder(props.data))
const activePoint = createMemo(() => props.data[activeIndex() ?? -1])
const metric = createMemo(() => props.metric ?? "tokens")
createEffect(() => scrollDenseChartToEnd(chartRef, props.range, props.data.length))
@@ -613,9 +618,10 @@ function TopModelsChart(props: {
ref={chartRef}
data-component="top-models-chart"
data-range={props.range}
data-metric={metric()}
data-dense-labels={isDenseColumnRange(props.range) ? "true" : undefined}
role="img"
aria-label="Stacked top model usage chart"
aria-label={props.ariaLabel ?? "Stacked top model usage chart"}
style={{ "--top-models-count": props.data.length } as JSX.CSSProperties}
onPointerLeave={(event) => {
if (event.pointerType === "touch") return
@@ -632,7 +638,7 @@ function TopModelsChart(props: {
data-mobile-hidden={isTopModelsMobileAxisHidden(index(), props.data.length) ? "true" : undefined}
>
<span data-slot="axis-label">
<span data-slot="axis-total">{formatTokens(usageTotal(day))}</span>
<span data-slot="axis-total">{formatUsageChartValue(usageTotal(day), metric())}</span>
<span data-slot="axis-date">
<span data-slot="axis-date-full">{day.date}</span>
<span data-slot="axis-date-mobile">{formatTopModelsMobileDate(day.date, props.range)}</span>
@@ -656,7 +662,7 @@ function TopModelsChart(props: {
data-slot="top-models-bar"
role="button"
tabIndex={0}
aria-label={`${day.date} ${formatTokens(usageTotal(day))}`}
aria-label={`${day.date} ${formatUsageChartValue(usageTotal(day), metric())} ${usageChartTotalLabel(metric())}`}
data-active={activeIndex() === dayIndex() ? "true" : undefined}
data-muted={activeIndex() !== undefined && activeIndex() !== dayIndex() ? "true" : undefined}
style={{ "--top-models-bar-height": `${getTopModelsBarHeight(usageTotal(day), maxTotal())}%` }}
@@ -738,7 +744,9 @@ function TopModelsChart(props: {
data-placement={dayIndex() > props.data.length * 0.62 ? "left" : "right"}
>
<strong>{point().date}</strong>
<span>{formatTokens(usageTotal(point()))} total</span>
<span>
{formatUsageChartValue(usageTotal(point()), metric())} {usageChartTotalLabel(metric())}
</span>
<div data-slot="tooltip-divider" />
<For each={visibleTopModelsSegments(point())}>
{(item) => (
@@ -758,7 +766,7 @@ function TopModelsChart(props: {
/>{" "}
{item.segment.model}
</span>
<b>{formatTokens(item.segment.value)}</b>
<b>{formatUsageChartValue(item.segment.value, metric())}</b>
</p>
)}
</For>
@@ -773,6 +781,33 @@ function TopModelsChart(props: {
)
}
function UniqueUsersSection(props: { data: StatsHomeData["users"] }) {
const [activeModel, setActiveModel] = createSignal<string>()
const data = createMemo(() => props.data.Go["2M"])
return (
<section id="unique-users" data-section="unique-users">
<SectionBridge label="TOP MODELS" href="#top-models" />
<SectionTitle title="Unique Users" description="Daily unique OpenCode Go users by model." />
<Show
when={data().some((item) => usageTotal(item) > 0)}
fallback={
<EmptyState title="No user data" description="No user-bearing model_stat rows matched this window." />
}
>
<TopModelsChart
data={data()}
range="2M"
metric="users"
ariaLabel="Stacked unique user chart by model"
activeModel={activeModel()}
onActiveModelChange={setActiveModel}
/>
</Show>
</section>
)
}
function isTopModelsBlankHover(bar: HTMLElement, clientY: number) {
const stack = bar.querySelector<HTMLElement>('[data-slot="top-models-stack"]')
if (!stack) return true
@@ -863,6 +898,22 @@ function formatTokens(value: number) {
return `${Math.round(value * 1000)}B`
}
function formatUsageChartValue(value: number, metric: "tokens" | "users") {
if (metric === "users") return formatUsers(value)
return formatTokens(value)
}
function usageChartTotalLabel(metric: "tokens" | "users") {
if (metric === "users") return "model users"
return "total"
}
function formatUsers(value: number) {
if (value >= 1_000_000) return `${(value / 1_000_000).toFixed(value >= 10_000_000 ? 0 : 1)}M`
if (value >= 1_000) return `${(value / 1_000).toFixed(value >= 10_000 ? 0 : 1)}K`
return new Intl.NumberFormat("en").format(Math.round(value))
}
function Leaderboard(props: {
data: LeaderboardEntry[]
activeModel: string | undefined
@@ -0,0 +1,3 @@
ALTER TABLE `geo_stat` ADD `unique_users` bigint NOT NULL DEFAULT 0;--> statement-breakpoint
ALTER TABLE `model_stat` ADD `unique_users` bigint NOT NULL DEFAULT 0;--> statement-breakpoint
ALTER TABLE `provider_stat` ADD `unique_users` bigint NOT NULL DEFAULT 0;
@@ -123,6 +123,7 @@ function metricColumns() {
return {
sessions: bigint({ mode: "number" }).notNull().default(0),
requests: bigint({ mode: "number" }).notNull().default(0),
unique_users: bigint({ mode: "number" }).notNull().default(0),
input_tokens: bigint({ mode: "number" }).notNull().default(0),
output_tokens: bigint({ mode: "number" }).notNull().default(0),
reasoning_tokens: bigint({ mode: "number" }).notNull().default(0),
+49 -36
View File
@@ -8,6 +8,8 @@ import {
chunks,
collapseRows,
inserted,
isMissingUniqueUsersColumn,
omitUniqueUsers,
rankRowsWithMarketShare,
statPeriodKey,
statRowScope,
@@ -136,48 +138,59 @@ export class GeoStatRepo extends Context.Service<GeoStatRepo, GeoStatRepo.Servic
chunks(rows, UPSERT_CHUNK_SIZE),
(chunk) =>
Effect.tryPromise({
try: () =>
db
.insert(geoStat)
.values(chunk)
.onDuplicateKeyUpdate({
set: {
continent: inserted("continent"),
sessions: inserted("sessions"),
requests: inserted("requests"),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
cache_read_tokens: inserted("cache_read_tokens"),
total_tokens: inserted("total_tokens"),
input_cost_microcents: inserted("input_cost_microcents"),
output_cost_microcents: inserted("output_cost_microcents"),
total_cost_microcents: inserted("total_cost_microcents"),
avg_duration_ms: inserted("avg_duration_ms"),
p50_duration_ms: inserted("p50_duration_ms"),
p95_duration_ms: inserted("p95_duration_ms"),
avg_ttfb_ms: inserted("avg_ttfb_ms"),
p50_ttfb_ms: inserted("p50_ttfb_ms"),
p95_ttfb_ms: inserted("p95_ttfb_ms"),
avg_output_tps: inserted("avg_output_tps"),
success_count: inserted("success_count"),
error_count: inserted("error_count"),
sample_count: inserted("sample_count"),
market_share_tokens: inserted("market_share_tokens"),
market_share_requests: inserted("market_share_requests"),
market_share_sessions: inserted("market_share_sessions"),
rank_by_tokens: inserted("rank_by_tokens"),
rank_by_requests: inserted("rank_by_requests"),
rank_by_sessions: inserted("rank_by_sessions"),
rank_by_cost: inserted("rank_by_cost"),
},
}),
try: async () => {
try {
return await upsertGeoChunk(chunk, true)
} catch (cause) {
if (!isMissingUniqueUsersColumn(cause)) throw cause
return upsertGeoChunk(chunk, false)
}
},
catch: (cause) => DatabaseError.make({ cause }),
}),
{ discard: true },
)
})
function upsertGeoChunk(chunk: GeoStatRow[], includeUniqueUsers: boolean) {
return db
.insert(geoStat)
.values(includeUniqueUsers ? chunk : omitUniqueUsers(chunk))
.onDuplicateKeyUpdate({
set: {
continent: inserted("continent"),
sessions: inserted("sessions"),
requests: inserted("requests"),
...(includeUniqueUsers ? { unique_users: inserted("unique_users") } : {}),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
cache_read_tokens: inserted("cache_read_tokens"),
total_tokens: inserted("total_tokens"),
input_cost_microcents: inserted("input_cost_microcents"),
output_cost_microcents: inserted("output_cost_microcents"),
total_cost_microcents: inserted("total_cost_microcents"),
avg_duration_ms: inserted("avg_duration_ms"),
p50_duration_ms: inserted("p50_duration_ms"),
p95_duration_ms: inserted("p95_duration_ms"),
avg_ttfb_ms: inserted("avg_ttfb_ms"),
p50_ttfb_ms: inserted("p50_ttfb_ms"),
p95_ttfb_ms: inserted("p95_ttfb_ms"),
avg_output_tps: inserted("avg_output_tps"),
success_count: inserted("success_count"),
error_count: inserted("error_count"),
sample_count: inserted("sample_count"),
market_share_tokens: inserted("market_share_tokens"),
market_share_requests: inserted("market_share_requests"),
market_share_sessions: inserted("market_share_sessions"),
rank_by_tokens: inserted("rank_by_tokens"),
rank_by_requests: inserted("rank_by_requests"),
rank_by_sessions: inserted("rank_by_sessions"),
rank_by_cost: inserted("rank_by_cost"),
},
})
}
const deleteRetiredDimensions = Effect.fn("GeoStatRepo.deleteRetiredDimensions")(function* (rows: GeoStatRow[]) {
const scope = statRowScope(rows)
if (!scope) return
+43 -14
View File
@@ -21,7 +21,7 @@ export type TokenCostEntry = { model: string; total: number; input: number; outp
export type CacheRatioEntry = { model: string; ratio: number; cached: number; uncached: number; total: number }
export type SessionCostEntry = { model: string; cost: number; tokens: number }
export type CountryEntry = { country: string; continent: string; tokens: number; share: number; rank: number }
export type ModelUsagePoint = { date: string; tokens: number; sessions: number; cost: number }
export type ModelUsagePoint = { date: string; tokens: number; users: number; sessions: number; cost: number }
export type ModelMixEntry = { label: string; tokens: number; share: number }
export type ModelPeerEntry = {
model: string
@@ -82,6 +82,7 @@ export type StatsLabData = {
export type StatsHomeData = {
updatedAt: string | null
usage: Record<UsageProduct, Record<UsageRange, UsagePoint[]>>
users: Record<UsageProduct, Record<UsageRange, UsagePoint[]>>
leaderboard: Record<UsageProduct, Record<UsageRange, LeaderboardEntry[]>>
market: Record<UsageRange, MarketDay[]>
tokenCost: Record<TokenProduct, TokenCostEntry[]>
@@ -118,6 +119,7 @@ type ModelAggregate = {
model: string
provider: string
sessions: number
uniqueUsers: number
inputTokens: number
outputTokens: number
reasoningTokens: number
@@ -200,6 +202,18 @@ function buildStatsHomeData(
),
),
),
users: createUsageProductRecord((product) =>
createRangeRecord((range) =>
buildUsagePoints(
normalized,
product,
range,
getWindow(range, earliest, latest),
getWindow("1W", earliest, latest),
"users",
),
),
),
leaderboard: createUsageProductRecord((product) =>
createRangeRecord((range) => buildLeaderboard(normalized, product, getWindow("1W", earliest, latest))),
),
@@ -248,15 +262,15 @@ function buildStatsModelData(
)
.filter((item) => item.totalTokens > 0)
.toSorted((a, b) => b.totalTokens - a.totalTokens || a.model.localeCompare(b.model))
const peers = aggregateByModelName(rowsForProduct(normalized, SITE_PRODUCT, window.start, window.end))
const windowPeers = aggregateByModelName(rowsForProduct(normalized, SITE_PRODUCT, window.start, window.end))
.filter((item) => item.totalTokens > 0)
.toSorted((a, b) => b.totalTokens - a.totalTokens || a.model.localeCompare(b.model))
const rankIndex = rankPeers.findIndex((item) => item.model === model)
const rank = rankIndex >= 0 ? rankIndex + 1 : null
const previousRankIndex = previousRankPeers.findIndex((item) => item.model === model)
const peerRankIndex = peers.findIndex((item) => item.model === model)
const peerRank = peerRankIndex >= 0 ? peerRankIndex + 1 : 1
const totalTokens = peers.reduce((sum, item) => sum + item.totalTokens, 0)
const peerRank = rankIndex >= 0 ? rankIndex + 1 : 1
const totalTokens = windowPeers.reduce((sum, item) => sum + item.totalTokens, 0)
const peerTokens = rankPeers.reduce((sum, item) => sum + item.totalTokens, 0)
return {
updatedAt: Number.isFinite(latestUpdate) ? new Date(latestUpdate).toISOString() : null,
@@ -266,7 +280,7 @@ function buildStatsModelData(
author: formatProvider(current.provider),
rank,
previousRank: previousRankIndex >= 0 ? previousRankIndex + 1 : null,
totalModels: peers.length,
totalModels: windowPeers.length,
tokenShare: totalTokens > 0 ? round((current.totalTokens / totalTokens) * 100, 2) : 0,
tokenChange: percentChange(current.totalTokens, previous.totalTokens),
totals: {
@@ -285,7 +299,7 @@ function buildStatsModelData(
usage: buildModelUsage(currentRows, window, "2M"),
tokenMix: buildModelTokenMix(current),
country: createRangeRecord((range) => buildCountryStats(geo, getWindow(range, earliest, latest))),
peers: buildModelPeers(peers, peerRank, totalTokens),
peers: buildModelPeers(rankPeers, peerRank, peerTokens),
}
}
@@ -340,6 +354,7 @@ function emptyStatsHomeData(): StatsHomeData {
return {
updatedAt: null,
usage: createUsageProductRecord(() => createRangeRecord(() => [])),
users: createUsageProductRecord(() => createRangeRecord(() => [])),
leaderboard: createUsageProductRecord(() => createRangeRecord(() => [])),
market: createRangeRecord(() => []),
tokenCost: createTokenProductRecord(() => []),
@@ -355,28 +370,39 @@ function buildUsagePoints(
range: UsageRange,
window: DateWindow,
rankWindow: DateWindow,
metric: "tokens" | "users" = "tokens",
) {
const modelOrder = aggregateByModelName(rowsForProduct(rows, product, rankWindow.start, rankWindow.end))
.toSorted((a, b) => b.totalTokens - a.totalTokens)
.toSorted((a, b) => modelUsageValue(b, metric) - modelUsageValue(a, metric))
.slice(0, TOP_MODEL_SEGMENT_LIMIT)
.map((item) => item.model)
return createBuckets(window, range).map((bucket) => {
const bucketRows = aggregateByModelName(rowsForProduct(rows, product, bucket.start, bucket.end))
const byModel = new Map(bucketRows.map((item) => [item.model, item.totalTokens]))
const segmentTokens = modelOrder.map((model) => ({ model, tokens: byModel.get(model) ?? 0 }))
const knownTokens = segmentTokens.reduce((sum, item) => sum + item.tokens, 0)
const totalTokens = bucketRows.reduce((sum, item) => sum + item.totalTokens, 0)
const byModel = new Map(bucketRows.map((item) => [item.model, modelUsageValue(item, metric)]))
const segments = modelOrder.map((model) => ({ model, value: byModel.get(model) ?? 0 }))
const knownValue = segments.reduce((sum, item) => sum + item.value, 0)
const totalValue = bucketRows.reduce((sum, item) => sum + modelUsageValue(item, metric), 0)
return {
date: bucket.label,
segments: [
...segmentTokens.map((item) => ({ model: item.model, value: round(item.tokens / 1_000_000_000_000, 4) })),
{ model: "Other", value: round(Math.max(totalTokens - knownTokens, 0) / 1_000_000_000_000, 4) },
...segments.map((item) => ({ model: item.model, value: usagePointValue(item.value, metric) })),
{ model: "Other", value: usagePointValue(Math.max(totalValue - knownValue, 0), metric) },
],
}
})
}
function modelUsageValue(item: ModelAggregate, metric: "tokens" | "users") {
if (metric === "users") return item.uniqueUsers
return item.totalTokens
}
function usagePointValue(value: number, metric: "tokens" | "users") {
if (metric === "users") return value
return round(value / 1_000_000_000_000, 4)
}
function buildLeaderboard(rows: StatMetricRow[], product: UsageProduct, rankWindow: DateWindow) {
const previous = new Map(
aggregateByModelName(rowsForProduct(rows, product, rankWindow.previousStart, rankWindow.previousEnd)).map(
@@ -502,6 +528,7 @@ function buildModelUsage(rows: StatMetricRow[], window: DateWindow, range: Usage
return {
date: bucket.label,
tokens: aggregate.totalTokens,
users: aggregate.uniqueUsers,
sessions: aggregate.sessions,
cost: round(microcentsToDollars(aggregate.totalCostMicrocents), 2),
}
@@ -601,6 +628,7 @@ function combineRowsForModel(model: string, rows: StatMetricRow[]): ModelAggrega
model,
provider: "unknown",
sessions: 0,
uniqueUsers: 0,
inputTokens: 0,
outputTokens: 0,
reasoningTokens: 0,
@@ -617,6 +645,7 @@ function combineModelAggregate(current: ModelAggregate | undefined, row: StatMet
model: row.model,
provider: row.provider,
sessions: (current?.sessions ?? 0) + row.sessions,
uniqueUsers: (current?.uniqueUsers ?? 0) + row.uniqueUsers,
inputTokens: (current?.inputTokens ?? 0) + row.inputTokens,
outputTokens: (current?.outputTokens ?? 0) + row.outputTokens,
reasoningTokens: (current?.reasoningTokens ?? 0) + row.reasoningTokens,
@@ -40,6 +40,7 @@ export function buildStatsQuery(periodStart: Date, periodEnd: Date, dimension: S
const aggregateColumns = `
COUNT(DISTINCT session) AS sessions,
COUNT(*) AS requests,
COUNT(DISTINCT user_key) AS unique_users,
COALESCE(SUM(tokens_input), 0) AS input_tokens,
COALESCE(SUM(tokens_output), 0) AS output_tokens,
COALESCE(SUM(tokens_reasoning), 0) AS reasoning_tokens,
@@ -70,6 +71,8 @@ WITH normalized AS (
UPPER(COALESCE(NULLIF(cf_country, ''), 'ZZ')) AS country,
COALESCE(NULLIF(cf_continent, ''), '') AS continent,
session,
COALESCE(NULLIF(workspace, ''), '') AS workspace,
COALESCE(NULLIF(api_key, ''), '') AS api_key,
status,
duration AS duration_ms,
time_to_first_byte AS ttfb_ms,
@@ -108,6 +111,7 @@ WITH normalized AS (
country,
continent,
session,
COALESCE(NULLIF(workspace, ''), NULLIF(api_key, '')) AS user_key,
status,
duration_ms,
ttfb_ms,
@@ -197,6 +201,7 @@ function toStatBaseAggregate(data: AthenaData): StatBaseAggregate[] {
tier: normalizeTier(data.tier || "unknown"),
sessions: integer(data, "sessions"),
requests: integer(data, "requests"),
unique_users: integer(data, "unique_users"),
input_tokens: integer(data, "input_tokens"),
output_tokens: integer(data, "output_tokens"),
reasoning_tokens: integer(data, "reasoning_tokens"),
+104 -60
View File
@@ -8,6 +8,8 @@ import {
chunks,
collapseRows,
inserted,
isMissingUniqueUsersColumn,
omitUniqueUsers,
rankBy,
statPeriodKey,
statRowScope,
@@ -27,6 +29,7 @@ export type ModelStatMetric = {
provider: string
model: string
sessions: number
uniqueUsers: number
inputTokens: number
outputTokens: number
reasoningTokens: number
@@ -55,34 +58,55 @@ export class ModelStatRepo extends Context.Service<ModelStatRepo, ModelStatRepo.
const listDaily = Effect.fn("ModelStatRepo.listDaily")(function* () {
return yield* Effect.tryPromise({
try: () =>
db
.select({
periodKey: modelStat.period_key,
updatedAt: modelStat.updated_at,
tier: modelStat.tier,
provider: modelStat.provider,
model: modelStat.model,
sessions: modelStat.sessions,
inputTokens: modelStat.input_tokens,
outputTokens: modelStat.output_tokens,
reasoningTokens: modelStat.reasoning_tokens,
cacheReadTokens: modelStat.cache_read_tokens,
totalTokens: modelStat.total_tokens,
inputCostMicrocents: modelStat.input_cost_microcents,
outputCostMicrocents: modelStat.output_cost_microcents,
totalCostMicrocents: modelStat.total_cost_microcents,
})
.from(modelStat)
.where(
and(
eq(modelStat.grain, "day"),
eq(modelStat.client, "all"),
eq(modelStat.source, "all"),
inArray(modelStat.tier, ["Go", "go"]),
),
)
.orderBy(asc(modelStat.period_key)),
try: async () => {
try {
return await db
.select({
periodKey: modelStat.period_key,
updatedAt: modelStat.updated_at,
tier: modelStat.tier,
provider: modelStat.provider,
model: modelStat.model,
sessions: modelStat.sessions,
uniqueUsers: modelStat.unique_users,
inputTokens: modelStat.input_tokens,
outputTokens: modelStat.output_tokens,
reasoningTokens: modelStat.reasoning_tokens,
cacheReadTokens: modelStat.cache_read_tokens,
totalTokens: modelStat.total_tokens,
inputCostMicrocents: modelStat.input_cost_microcents,
outputCostMicrocents: modelStat.output_cost_microcents,
totalCostMicrocents: modelStat.total_cost_microcents,
})
.from(modelStat)
.where(modelDailyScope())
.orderBy(asc(modelStat.period_key))
} catch (cause) {
if (!isMissingUniqueUsersColumn(cause)) throw cause
return (
await db
.select({
periodKey: modelStat.period_key,
updatedAt: modelStat.updated_at,
tier: modelStat.tier,
provider: modelStat.provider,
model: modelStat.model,
sessions: modelStat.sessions,
inputTokens: modelStat.input_tokens,
outputTokens: modelStat.output_tokens,
reasoningTokens: modelStat.reasoning_tokens,
cacheReadTokens: modelStat.cache_read_tokens,
totalTokens: modelStat.total_tokens,
inputCostMicrocents: modelStat.input_cost_microcents,
outputCostMicrocents: modelStat.output_cost_microcents,
totalCostMicrocents: modelStat.total_cost_microcents,
})
.from(modelStat)
.where(modelDailyScope())
.orderBy(asc(modelStat.period_key))
).map((row) => ({ ...row, uniqueUsers: 0 }))
}
},
catch: (cause) => DatabaseError.make({ cause }),
})
})
@@ -92,44 +116,55 @@ export class ModelStatRepo extends Context.Service<ModelStatRepo, ModelStatRepo.
chunks(rows, UPSERT_CHUNK_SIZE),
(chunk) =>
Effect.tryPromise({
try: () =>
db
.insert(modelStat)
.values(chunk)
.onDuplicateKeyUpdate({
set: {
provider_model: inserted("provider_model"),
sessions: inserted("sessions"),
requests: inserted("requests"),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
cache_read_tokens: inserted("cache_read_tokens"),
total_tokens: inserted("total_tokens"),
input_cost_microcents: inserted("input_cost_microcents"),
output_cost_microcents: inserted("output_cost_microcents"),
total_cost_microcents: inserted("total_cost_microcents"),
avg_duration_ms: inserted("avg_duration_ms"),
p50_duration_ms: inserted("p50_duration_ms"),
p95_duration_ms: inserted("p95_duration_ms"),
avg_ttfb_ms: inserted("avg_ttfb_ms"),
p50_ttfb_ms: inserted("p50_ttfb_ms"),
p95_ttfb_ms: inserted("p95_ttfb_ms"),
avg_output_tps: inserted("avg_output_tps"),
success_count: inserted("success_count"),
error_count: inserted("error_count"),
sample_count: inserted("sample_count"),
rank_by_tokens: inserted("rank_by_tokens"),
rank_by_requests: inserted("rank_by_requests"),
rank_by_cost: inserted("rank_by_cost"),
},
}),
try: async () => {
try {
return await upsertModelChunk(chunk, true)
} catch (cause) {
if (!isMissingUniqueUsersColumn(cause)) throw cause
return upsertModelChunk(chunk, false)
}
},
catch: (cause) => DatabaseError.make({ cause }),
}),
{ discard: true },
)
})
function upsertModelChunk(chunk: ModelStatRow[], includeUniqueUsers: boolean) {
return db
.insert(modelStat)
.values(includeUniqueUsers ? chunk : omitUniqueUsers(chunk))
.onDuplicateKeyUpdate({
set: {
provider_model: inserted("provider_model"),
sessions: inserted("sessions"),
requests: inserted("requests"),
...(includeUniqueUsers ? { unique_users: inserted("unique_users") } : {}),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
cache_read_tokens: inserted("cache_read_tokens"),
total_tokens: inserted("total_tokens"),
input_cost_microcents: inserted("input_cost_microcents"),
output_cost_microcents: inserted("output_cost_microcents"),
total_cost_microcents: inserted("total_cost_microcents"),
avg_duration_ms: inserted("avg_duration_ms"),
p50_duration_ms: inserted("p50_duration_ms"),
p95_duration_ms: inserted("p95_duration_ms"),
avg_ttfb_ms: inserted("avg_ttfb_ms"),
p50_ttfb_ms: inserted("p50_ttfb_ms"),
p95_ttfb_ms: inserted("p95_ttfb_ms"),
avg_output_tps: inserted("avg_output_tps"),
success_count: inserted("success_count"),
error_count: inserted("error_count"),
sample_count: inserted("sample_count"),
rank_by_tokens: inserted("rank_by_tokens"),
rank_by_requests: inserted("rank_by_requests"),
rank_by_cost: inserted("rank_by_cost"),
},
})
}
const deleteRetiredDimensions = Effect.fn("ModelStatRepo.deleteRetiredDimensions")(function* (
rows: ModelStatRow[],
) {
@@ -162,6 +197,15 @@ export class ModelStatRepo extends Context.Service<ModelStatRepo, ModelStatRepo.
)
}
function modelDailyScope() {
return and(
eq(modelStat.grain, "day"),
eq(modelStat.client, "all"),
eq(modelStat.source, "all"),
inArray(modelStat.tier, ["Go", "go"]),
)
}
export function rowsFromAggregates(aggregates: ModelStatAggregate[]) {
return rankRows([
...synthesizeAllTierRows(
+48 -35
View File
@@ -8,6 +8,8 @@ import {
chunks,
collapseRows,
inserted,
isMissingUniqueUsersColumn,
omitUniqueUsers,
rankRowsWithMarketShare,
statRowScope,
synthesizeAllTierRows,
@@ -107,47 +109,58 @@ export class ProviderStatRepo extends Context.Service<ProviderStatRepo, Provider
chunks(rows, UPSERT_CHUNK_SIZE),
(chunk) =>
Effect.tryPromise({
try: () =>
db
.insert(providerStat)
.values(chunk)
.onDuplicateKeyUpdate({
set: {
sessions: inserted("sessions"),
requests: inserted("requests"),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
cache_read_tokens: inserted("cache_read_tokens"),
total_tokens: inserted("total_tokens"),
input_cost_microcents: inserted("input_cost_microcents"),
output_cost_microcents: inserted("output_cost_microcents"),
total_cost_microcents: inserted("total_cost_microcents"),
avg_duration_ms: inserted("avg_duration_ms"),
p50_duration_ms: inserted("p50_duration_ms"),
p95_duration_ms: inserted("p95_duration_ms"),
avg_ttfb_ms: inserted("avg_ttfb_ms"),
p50_ttfb_ms: inserted("p50_ttfb_ms"),
p95_ttfb_ms: inserted("p95_ttfb_ms"),
avg_output_tps: inserted("avg_output_tps"),
success_count: inserted("success_count"),
error_count: inserted("error_count"),
sample_count: inserted("sample_count"),
market_share_tokens: inserted("market_share_tokens"),
market_share_requests: inserted("market_share_requests"),
market_share_sessions: inserted("market_share_sessions"),
rank_by_tokens: inserted("rank_by_tokens"),
rank_by_requests: inserted("rank_by_requests"),
rank_by_sessions: inserted("rank_by_sessions"),
rank_by_cost: inserted("rank_by_cost"),
},
}),
try: async () => {
try {
return await upsertProviderChunk(chunk, true)
} catch (cause) {
if (!isMissingUniqueUsersColumn(cause)) throw cause
return upsertProviderChunk(chunk, false)
}
},
catch: (cause) => DatabaseError.make({ cause }),
}),
{ discard: true },
)
})
function upsertProviderChunk(chunk: ProviderStatRow[], includeUniqueUsers: boolean) {
return db
.insert(providerStat)
.values(includeUniqueUsers ? chunk : omitUniqueUsers(chunk))
.onDuplicateKeyUpdate({
set: {
sessions: inserted("sessions"),
requests: inserted("requests"),
...(includeUniqueUsers ? { unique_users: inserted("unique_users") } : {}),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
cache_read_tokens: inserted("cache_read_tokens"),
total_tokens: inserted("total_tokens"),
input_cost_microcents: inserted("input_cost_microcents"),
output_cost_microcents: inserted("output_cost_microcents"),
total_cost_microcents: inserted("total_cost_microcents"),
avg_duration_ms: inserted("avg_duration_ms"),
p50_duration_ms: inserted("p50_duration_ms"),
p95_duration_ms: inserted("p95_duration_ms"),
avg_ttfb_ms: inserted("avg_ttfb_ms"),
p50_ttfb_ms: inserted("p50_ttfb_ms"),
p95_ttfb_ms: inserted("p95_ttfb_ms"),
avg_output_tps: inserted("avg_output_tps"),
success_count: inserted("success_count"),
error_count: inserted("error_count"),
sample_count: inserted("sample_count"),
market_share_tokens: inserted("market_share_tokens"),
market_share_requests: inserted("market_share_requests"),
market_share_sessions: inserted("market_share_sessions"),
rank_by_tokens: inserted("rank_by_tokens"),
rank_by_requests: inserted("rank_by_requests"),
rank_by_sessions: inserted("rank_by_sessions"),
rank_by_cost: inserted("rank_by_cost"),
},
})
}
const deleteRetiredDimensions = Effect.fn("ProviderStatRepo.deleteRetiredDimensions")(function* (
rows: ProviderStatRow[],
) {
+25
View File
@@ -12,6 +12,7 @@ export type StatBaseAggregate = {
tier: string
sessions: number
requests: number
unique_users: number
input_tokens: number
output_tokens: number
reasoning_tokens: number
@@ -41,6 +42,7 @@ export type StatBaseRow = {
source?: string
sessions?: number
requests?: number
unique_users?: number
input_tokens?: number
output_tokens?: number
reasoning_tokens?: number
@@ -71,6 +73,7 @@ export function toStatBaseRow(data: StatBaseAggregate) {
source: "all",
sessions: data.sessions,
requests: data.requests,
unique_users: data.unique_users,
input_tokens: data.input_tokens,
output_tokens: data.output_tokens,
reasoning_tokens: data.reasoning_tokens,
@@ -122,6 +125,7 @@ export function combineRows<T extends StatBaseRow>(left: T, right: T): T {
...left,
sessions: (left.sessions ?? 0) + (right.sessions ?? 0),
requests: (left.requests ?? 0) + (right.requests ?? 0),
unique_users: (left.unique_users ?? 0) + (right.unique_users ?? 0),
input_tokens: (left.input_tokens ?? 0) + (right.input_tokens ?? 0),
output_tokens: (left.output_tokens ?? 0) + (right.output_tokens ?? 0),
reasoning_tokens: (left.reasoning_tokens ?? 0) + (right.reasoning_tokens ?? 0),
@@ -143,6 +147,18 @@ export function combineRows<T extends StatBaseRow>(left: T, right: T): T {
}
}
export function isMissingUniqueUsersColumn(cause: unknown): boolean {
return errorText(cause).includes("Unknown column 'unique_users'")
}
export function omitUniqueUsers<T extends { unique_users?: number }>(rows: T[]) {
return rows.map((row) => {
const result = { ...row }
delete result.unique_users
return result
})
}
export function statPeriodKey(row: StatBaseRow) {
return [row.grain, row.period_key, row.dataset, row.tier, row.client, row.source].join("\u0000")
}
@@ -238,6 +254,15 @@ export function inserted(column: string) {
return sql.raw(`values(\`${column}\`)`)
}
function errorText(cause: unknown): string {
if (cause instanceof Error) return `${cause.message} ${errorText((cause as { cause?: unknown }).cause)}`
if (typeof cause === "object" && cause)
return Object.values(cause as Record<string, unknown>)
.map(errorText)
.join(" ")
return String(cause)
}
export function weightedAverage(
left: number | null | undefined,
leftWeight = 0,
+14 -3
View File
@@ -242,6 +242,7 @@ function metricQuery(breakdowns: string[], limit: number, filters: ReturnType<ty
calculations: [
{ op: "COUNT_DISTINCT", column: "session" },
{ op: "COUNT" },
{ op: "COUNT_DISTINCT", column: "workspace" },
{ op: "SUM", column: "tokens.input" },
{ op: "SUM", column: "tokens.output" },
{ op: "SUM", column: "tokens.reasoning" },
@@ -374,9 +375,15 @@ function classifyRows(file: string, rows: RawRow[]): ImportKey {
}
function hasMetricHeaders(headers: Set<string>) {
return ["sumtokens", "sumtokensinput", "inputtokens", "totaltokens", "avgduration", "countdistinctsession"].some(
(header) => headers.has(header),
)
return [
"sumtokens",
"sumtokensinput",
"inputtokens",
"totaltokens",
"avgduration",
"countdistinctsession",
"countdistinctworkspace",
].some((header) => headers.has(header))
}
function hasHeader(headers: Set<string>, names: string[]) {
@@ -447,6 +454,7 @@ function baseAggregate(row: RawRow, grain: Grain, opts: ImportOptions): StatBase
tier: tier(row),
sessions: integer(row, "sessions", ["COUNT_DISTINCT(session)"]),
requests: integer(row, "requests", ["COUNT", "COUNT()"]),
unique_users: integer(row, "unique_users", ["COUNT_DISTINCT(workspace)", "COUNT_DISTINCT(api_key)"]),
input_tokens: integer(row, "input_tokens", ["SUM(tokens.input)", "SUM(tokens_input)"]),
output_tokens: integer(row, "output_tokens", ["SUM(tokens.output)", "SUM(tokens_output)"]),
reasoning_tokens: integer(row, "reasoning_tokens", ["SUM(tokens.reasoning)", "SUM(tokens_reasoning)"]),
@@ -808,6 +816,7 @@ async function upsertModelRows(db: ReturnType<typeof drizzle>, rows: ModelStatRo
provider_model: inserted("provider_model"),
sessions: inserted("sessions"),
requests: inserted("requests"),
unique_users: inserted("unique_users"),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
@@ -845,6 +854,7 @@ async function upsertProviderRows(db: ReturnType<typeof drizzle>, rows: Provider
set: {
sessions: inserted("sessions"),
requests: inserted("requests"),
unique_users: inserted("unique_users"),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
@@ -887,6 +897,7 @@ async function upsertGeoRows(db: ReturnType<typeof drizzle>, rows: GeoStatRow[],
continent: inserted("continent"),
sessions: inserted("sessions"),
requests: inserted("requests"),
unique_users: inserted("unique_users"),
input_tokens: inserted("input_tokens"),
output_tokens: inserted("output_tokens"),
reasoning_tokens: inserted("reasoning_tokens"),
+7 -1
View File
@@ -11,6 +11,7 @@ import { startOfIsoWeek } from "./domain/stat"
const DATALAKE_INGESTION_LAG_MS = 5 * 60_000
const STATS_DATA_START_MS = new Date("2026-05-28T00:00:00.000Z").getTime()
const WEEK_MS = 7 * 86_400_000
const DISPLAY_WINDOW_MS = 56 * 86_400_000
export type SyncStatsResult = { ok: true; rows: number; startedAt: string; periodStart: string; periodEnd: string }
export type SyncStatsError = AthenaQueryError | AthenaQueryTimeoutError | DatabaseError
@@ -23,7 +24,12 @@ export const syncStats: () => Effect.Effect<
const startedAt = yield* DateTime.nowAsDate
const periodEnd = new Date(Math.floor((startedAt.getTime() - DATALAKE_INGESTION_LAG_MS) / 60_000) * 60_000)
// May 27 was partial, so keep Athena stats anchored at the first complete day.
const periodStart = new Date(Math.max(startOfIsoWeek(periodEnd).getTime() - WEEK_MS, STATS_DATA_START_MS))
const periodStart = new Date(
Math.max(
Math.min(startOfIsoWeek(periodEnd).getTime() - WEEK_MS, periodEnd.getTime() - DISPLAY_WINDOW_MS),
STATS_DATA_START_MS,
),
)
const athena = yield* Athena
const modelStats = yield* ModelStatRepo
const providerStats = yield* ProviderStatRepo
@@ -1,9 +1,11 @@
import { createResource, createMemo } from "solid-js"
import { createResource, createMemo, createSignal } from "solid-js"
import { TextAttributes } from "@opentui/core"
import { DialogSelect } from "../ui/dialog-select"
import { useSDK } from "../context/sdk"
import { useDialog } from "../ui/dialog"
import { useToast } from "../ui/toast"
import { useTheme } from "../context/theme"
import { errorMessage } from "../util/error"
import type { ExperimentalConsoleListOrgsResponse } from "@opencode-ai/sdk/v2"
type OrgOption = ExperimentalConsoleListOrgsResponse["orgs"][number]
@@ -25,14 +27,26 @@ export function DialogConsoleOrg() {
const toast = useToast()
const { theme } = useTheme()
const [orgs] = createResource(async () => {
const result = await sdk.client.experimental.console.listOrgs({}, { throwOnError: true })
return result.data?.orgs ?? []
})
const [loadError, setLoadError] = createSignal<unknown>()
const [orgs] = createResource(() =>
sdk.client.experimental.console
.listOrgs({}, { throwOnError: true })
.then((result) => result.data?.orgs ?? [])
// Catch so the rejected resource never reaches the memos below: reading
// orgs() in an errored state re-throws and tears down the dialog.
.catch((error) => {
setLoadError(error)
return undefined
}),
)
const showError = createMemo(() => Boolean(loadError()))
const current = createMemo(() => orgs()?.find((item) => item.active))
const options = createMemo(() => {
if (showError()) return []
const listed = orgs()
if (listed === undefined) {
return [
@@ -99,5 +113,23 @@ export function DialogConsoleOrg() {
}))
})
return <DialogSelect<string | OrgOption> title="Switch org" options={options()} current={current()} />
return (
<DialogSelect<string | OrgOption>
title="Switch org"
options={options()}
current={current()}
renderFilter={!showError()}
locked={showError()}
emptyView={
showError() ? (
<box paddingLeft={4} paddingRight={4}>
<text fg={theme.error} attributes={TextAttributes.BOLD}>
Could not load orgs
</text>
<text fg={theme.textMuted}>{errorMessage(loadError())}</text>
</box>
) : undefined
}
/>
)
}
+11 -36
View File
@@ -91,6 +91,8 @@ const GO_UPSELL_ACCOUNT_RATE_LIMIT_DONT_SHOW = "go_upsell_account_rate_limit_don
const GO_UPSELL_WINDOW = 86_400_000 // 24 hrs
const GO_UPSELL_PROVIDERS = new Set(["opencode", "opencode-go"])
export const alwaysSeparate = new WeakSet<BoxRenderable>()
type RetryAction = Extract<SessionStatus, { type: "retry" }>["action"]
function goUpsellKeys(action: RetryAction) {
@@ -160,7 +162,6 @@ const context = createContext<{
showTimestamps: () => boolean
showDetails: () => boolean
showGenericToolOutput: () => boolean
userMessageIDs: () => ReadonlySet<string>
diffWrapMode: () => "word" | "none"
providers: () => ReadonlyMap<string, Provider>
sync: ReturnType<typeof useSync>
@@ -218,14 +219,6 @@ export function Session() {
)
: [],
)
const userMessageIDs = createMemo(
() =>
new Set(
messages()
.filter((message) => message.role === "user")
.map((message) => message.id),
),
)
const permissions = createMemo(() => {
if (session()?.parentID) return []
return children().flatMap((x) => sync.data.permission[x.id] ?? [])
@@ -1158,7 +1151,6 @@ export function Session() {
showTimestamps,
showDetails,
showGenericToolOutput,
userMessageIDs,
diffWrapMode,
providers,
sync,
@@ -1395,6 +1387,7 @@ function UserMessage(props: {
<Show when={text()}>
<box
id={props.message.id}
ref={(el: BoxRenderable) => alwaysSeparate.add(el)}
border={["left"]}
borderColor={color()}
customBorderChars={SplitBorder.customBorderChars}
@@ -1532,7 +1525,7 @@ function AssistantMessage(props: { message: AssistantMessage; parts: Part[]; las
</Show>
<Show when={props.message.error && props.message.error.name !== "MessageAbortedError"}>
<box
id={`assistant-error-${props.message.id}`}
ref={(el: BoxRenderable) => alwaysSeparate.add(el)}
border={["left"]}
paddingTop={1}
paddingBottom={1}
@@ -1547,7 +1540,7 @@ function AssistantMessage(props: { message: AssistantMessage; parts: Part[]; las
</Show>
<Switch>
<Match when={props.last || final() || props.message.error?.name === "MessageAbortedError"}>
<box id={`assistant-summary-${props.message.id}`} paddingLeft={3}>
<box ref={(el: BoxRenderable) => alwaysSeparate.add(el)} paddingLeft={3}>
<text marginTop={1}>
<span
style={{
@@ -1613,7 +1606,7 @@ function ReasoningPart(props: { last: boolean; part: ReasoningPart; message: Ass
return (
<Show when={content()}>
<box
id={`text-${props.part.messageID}-${props.part.id}`}
ref={(el: BoxRenderable) => alwaysSeparate.add(el)}
paddingLeft={3}
marginTop={1}
flexDirection="column"
@@ -1695,7 +1688,7 @@ function TextPart(props: { last: boolean; part: TextPart; message: AssistantMess
const { theme, syntax } = useTheme()
return (
<Show when={props.part.text.trim()}>
<box id={`text-${props.part.messageID}-${props.part.id}`} paddingLeft={3} marginTop={1} flexShrink={0}>
<box ref={(el: BoxRenderable) => alwaysSeparate.add(el)} paddingLeft={3} marginTop={1} flexShrink={0}>
<markdown
syntaxStyle={syntax()}
streaming={true}
@@ -1845,7 +1838,6 @@ function InlineTool(props: {
pending: string
failure?: string
spinner?: boolean
subagent?: boolean
children: JSX.Element
part: ToolPart
onClick?: () => void
@@ -1886,7 +1878,6 @@ function InlineTool(props: {
return (
<InlineToolRow
id={`tool-inline-${props.subagent ? "subagent-" : ""}${props.part.messageID}-${props.part.id}`}
icon={props.icon}
iconColor={props.iconColor}
color={fg()}
@@ -1899,8 +1890,6 @@ function InlineTool(props: {
pending={props.pending}
failure={props.failure}
spinner={props.spinner}
subagent={props.subagent}
separateAfter={(id) => id !== undefined && ctx.userMessageIDs().has(id)}
onMouseOver={() => clickable() && setHover(true)}
onMouseOut={() => setHover(false)}
onMouseUp={() => {
@@ -1918,7 +1907,6 @@ function InlineTool(props: {
}
export function InlineToolRow(props: {
id?: string
icon: string
iconColor?: RGBA
color?: RGBA
@@ -1931,32 +1919,20 @@ export function InlineToolRow(props: {
pending: string
failure?: string
spinner?: boolean
subagent?: boolean
children: JSX.Element
separateAfter?: (id: string | undefined) => boolean
onMouseOver?: () => void
onMouseOut?: () => void
onMouseUp?: () => void
}) {
return (
<box
id={props.id}
paddingLeft={3}
onMouseOver={props.onMouseOver}
onMouseOut={props.onMouseOut}
onMouseUp={props.onMouseUp}
ref={(el: BoxRenderable) => {
setPreLayoutSiblingMargin(el, (previous) => {
const previousInline = previous?.id.startsWith("tool-inline-") ?? false
const previousSubagent = previous?.id.startsWith("tool-inline-subagent-") ?? false
return previous?.id.startsWith("text-") ||
previous?.id.startsWith("tool-block-") ||
previous?.id.startsWith("assistant-error-") ||
previous?.id.startsWith("assistant-summary-") ||
(previousInline && previousSubagent !== Boolean(props.subagent)) ||
props.separateAfter?.(previous?.id)
? 1
: 0
return previous instanceof BoxRenderable && (previous.height > 1 || alwaysSeparate.has(previous)) ? 1 : 0
})
}}
>
@@ -2018,7 +1994,7 @@ function BlockTool(props: {
const error = createMemo(() => (props.part?.state.status === "error" ? props.part.state.error : undefined))
return (
<box
id={props.part ? `tool-block-${props.part.messageID}-${props.part.id}` : undefined}
ref={(el: BoxRenderable) => alwaysSeparate.add(el)}
border={["left"]}
paddingTop={1}
paddingBottom={1}
@@ -2184,8 +2160,8 @@ function Read(props: ToolProps) {
Read {pathFormatter.format(stringValue(props.input.filePath))} {input(props.input, ["filePath"])}
</InlineTool>
<For each={loaded()}>
{(filepath, index) => (
<box id={`tool-inline-loaded-${props.part.messageID}-${props.part.id}-${index()}`} paddingLeft={3}>
{(filepath) => (
<box paddingLeft={3}>
<text paddingLeft={3} fg={theme.textMuted}>
Loaded {pathFormatter.format(filepath)}
</text>
@@ -2305,7 +2281,6 @@ function Task(props: ToolProps) {
return (
<InlineTool
icon={props.part.state.status === "completed" ? "✓" : "│"}
subagent={true}
color={retry() ? theme.error : undefined}
spinner={isRunning()}
complete={stringValue(props.input.description)}
@@ -3,6 +3,7 @@
exports[`TUI inline tool wrapping snapshots consecutive grep, glob, and read rows at a narrow width 1`] = `
" ✱ Grep "OPENCODE.*DB|database|sqlite|drizzle|dev.*db|data.
*dir|xdg|APPDATA" in packages/opencode/src (151 matches)
✱ Glob "**/*db*" in packages/opencode (6 matches)
→ Read packages/opencode/src/storage/db.ts [offset=1, limit=130]
→ Read packages/opencode/src/index.ts [offset=1, limit=100]
@@ -13,10 +14,12 @@ exports[`TUI inline tool wrapping snapshots consecutive grep, glob, and read row
exports[`TUI inline tool wrapping snapshots expanded tool errors under the tool text 1`] = `
" ✱ Grep "OPENCODE.*DB|database|sqlite|drizzle|dev.*db|data.
*dir|xdg|APPDATA" in packages/opencode/src (151 matches)
✱ Glob "**/*db*" in packages/opencode (6 matches)
→ Read packages/opencode/src/storage/db.ts [offset=1, limit=130]
→ Read packages/opencode/src/index.ts [offset=1, limit=100]
No LSP server available for this file type.
✱ Grep "export const OPENCODE_DB|OPENCODE_DB|OPENCODE_DEV|Global\\.
Path\\.data|data =" in packages/opencode/src (115 matches)"
`;
@@ -32,6 +35,7 @@ exports[`TUI inline tool wrapping keeps separation after a shell output block 1`
✱ Grep "OPENCODE.*DB|database|sqlite|drizzle|dev.*db|data.
*dir|xdg|APPDATA" in packages/opencode/src (151 matches)
✱ Glob "**/*db*" in packages/opencode (6 matches)
→ Read packages/opencode/src/storage/db.ts [offset=1, limit=130]
→ Read packages/opencode/src/index.ts [offset=1, limit=100]
@@ -46,6 +50,7 @@ exports[`TUI inline tool wrapping keeps separation after a padded user message 1
✱ Grep "OPENCODE.*DB|database|sqlite|drizzle|dev.*db|data.
*dir|xdg|APPDATA" in packages/opencode/src (151 matches)
✱ Glob "**/*db*" in packages/opencode (6 matches)
→ Read packages/opencode/src/storage/db.ts [offset=1, limit=130]
→ Read packages/opencode/src/index.ts [offset=1, limit=100]
@@ -53,9 +58,8 @@ exports[`TUI inline tool wrapping keeps separation after a padded user message 1
Path\\.data|data =" in packages/opencode/src (115 matches)"
`;
exports[`TUI inline tool wrapping separates a contiguous subagent group from inline tools 1`] = `
exports[`TUI inline tool wrapping separates after a multi-line task row 1`] = `
" ✱ Grep "Task" (2 matches)
⠙ Explore Task — Inspect active task spacing
✓ General Task — Confirm completed task spacing
↳ 1 toolcall · 501ms
@@ -63,22 +67,21 @@ exports[`TUI inline tool wrapping separates a contiguous subagent group from inl
→ Read src/cli/cmd/tui/routes/session/index.tsx"
`;
exports[`TUI inline tool wrapping separates a subagent group after an expanded read 1`] = `
exports[`TUI inline tool wrapping does not treat task rows differently from other inline rows 1`] = `
" → Read src/cli/cmd/tui/routes/session/index.tsx
↳ Loaded src/cli/cmd/tui/routes/session/tools.tsx
✓ Explore Task — Inspect active task spacing
↳ 1 toolcall · 501ms"
`;
exports[`TUI inline tool wrapping separates a subagent from the previous assistant summary 1`] = `
exports[`TUI inline tool wrapping separates an inline row from the previous assistant summary 1`] = `
" ▣ Build · Little Frank · 53.1s
✓ Build Task — Review changes
↳ 48 toolcalls · 1m 40s"
`;
exports[`TUI inline tool wrapping separates a subagent from the previous assistant error 1`] = `
exports[`TUI inline tool wrapping separates an inline row from the previous assistant error 1`] = `
"│
│ Managed inference requires an active Member plan
@@ -1,6 +1,6 @@
import { afterEach, describe, expect, test } from "bun:test"
import { createSignal, For, Show } from "solid-js"
import type { ScrollBoxRenderable } from "@opentui/core"
import type { BoxRenderable, ScrollBoxRenderable } from "@opentui/core"
import { testRender, type JSX } from "@opentui/solid"
import {
formatCompletedSubagentDetail,
@@ -13,6 +13,7 @@ import {
parseQuestionAnswers,
parseQuestions,
parseTodos,
alwaysSeparate,
toolDisplay,
} from "../../../src/routes/session"
@@ -53,7 +54,14 @@ const tools: readonly ToolFixture[] = [
function ShellOutput() {
return (
<box id="tool-block-shell" marginTop={1} paddingTop={1} paddingBottom={1} paddingLeft={2} gap={1}>
<box
ref={(el: BoxRenderable) => alwaysSeparate.add(el)}
marginTop={1}
paddingTop={1}
paddingBottom={1}
paddingLeft={2}
gap={1}
>
<text paddingLeft={3}># List files</text>
<box gap={1}>
<text>$ ls</text>
@@ -65,7 +73,7 @@ function ShellOutput() {
function UserMessage() {
return (
<box id="message-user">
<box ref={(el: BoxRenderable) => alwaysSeparate.add(el)}>
<box paddingTop={1} paddingBottom={1} paddingLeft={2}>
<text>Check whether the next tool remains separated.</text>
</box>
@@ -88,7 +96,6 @@ function Fixture(props: { errorExpanded?: boolean; before?: "shell" | "user" })
failed={Boolean(item.error)}
error={item.error}
errorExpanded={props.errorExpanded}
separateAfter={(id) => id === "message-user"}
>
{item.label}
</InlineToolRow>
@@ -99,61 +106,67 @@ function Fixture(props: { errorExpanded?: boolean; before?: "shell" | "user" })
)
}
function SubagentGroupFixture() {
function TaskRowsFixture() {
return (
<box flexDirection="column" width={72}>
<InlineToolRow id="tool-inline-before" icon="✱" complete={true} pending="">
<InlineToolRow icon="✱" complete={true} pending="">
Grep "Task" (2 matches)
</InlineToolRow>
<InlineToolRow id="tool-inline-subagent-one" icon="⠙" complete={true} pending="" subagent={true}>
<InlineToolRow icon="⠙" complete={true} pending="">
Explore Task Inspect active task spacing
</InlineToolRow>
<InlineToolRow id="tool-inline-subagent-two" icon="✓" complete={true} pending="" subagent={true}>
<InlineToolRow icon="✓" complete={true} pending="">
{"General Task — Confirm completed task spacing\n↳ 1 toolcall · 501ms"}
</InlineToolRow>
<InlineToolRow id="tool-inline-after" icon="→" complete={true} pending="">
<InlineToolRow icon="→" complete={true} pending="">
Read src/cli/cmd/tui/routes/session/index.tsx
</InlineToolRow>
</box>
)
}
function LoadedReadBeforeSubagentFixture() {
function LoadedReadBeforeTaskFixture() {
return (
<box flexDirection="column" width={72}>
<InlineToolRow id="tool-inline-read" icon="→" complete={true} pending="">
<InlineToolRow icon="→" complete={true} pending="">
Read src/cli/cmd/tui/routes/session/index.tsx
</InlineToolRow>
<box id="tool-inline-loaded-read-child" paddingLeft={3}>
<box paddingLeft={3}>
<text paddingLeft={3}> Loaded src/cli/cmd/tui/routes/session/tools.tsx</text>
</box>
<InlineToolRow id="tool-inline-subagent-after-read" icon="✓" complete={true} pending="" subagent={true}>
<InlineToolRow icon="✓" complete={true} pending="">
{"Explore Task — Inspect active task spacing\n↳ 1 toolcall · 501ms"}
</InlineToolRow>
</box>
)
}
function AssistantSummaryBeforeSubagentFixture() {
function AssistantSummaryBeforeInlineFixture() {
return (
<box flexDirection="column" width={72}>
<box id="assistant-summary-message-one" paddingLeft={3}>
<box ref={(el: BoxRenderable) => alwaysSeparate.add(el)} paddingLeft={3}>
<text> Build · Little Frank · 53.1s</text>
</box>
<InlineToolRow id="tool-inline-subagent-one" icon="✓" complete={true} pending="" subagent={true}>
<InlineToolRow icon="✓" complete={true} pending="">
{"Build Task — Review changes\n↳ 48 toolcalls · 1m 40s"}
</InlineToolRow>
</box>
)
}
function AssistantErrorBeforeSubagentFixture() {
function AssistantErrorBeforeInlineFixture() {
return (
<box flexDirection="column" width={72}>
<box id="assistant-error-message-one" border={["left"]} paddingTop={1} paddingBottom={1} paddingLeft={2}>
<box
ref={(el: BoxRenderable) => alwaysSeparate.add(el)}
border={["left"]}
paddingTop={1}
paddingBottom={1}
paddingLeft={2}
>
<text>Managed inference requires an active Member plan</text>
</box>
<InlineToolRow id="tool-inline-subagent-one" icon="✓" complete={true} pending="" subagent={true}>
<InlineToolRow icon="✓" complete={true} pending="">
{"Build Task — Review changes\n↳ 48 toolcalls · 1m 40s"}
</InlineToolRow>
</box>
@@ -170,7 +183,7 @@ function StickyScrollFixture(props: { separated: boolean; scroll: (scroll: Scrol
<text>Second row</text>
</box>
<Show when={props.separated}>
<box id="text-before-tool">
<box ref={(el: BoxRenderable) => alwaysSeparate.add(el)}>
<text>Assistant text</text>
</box>
</Show>
@@ -200,6 +213,7 @@ function FailedCompleteToolFixture() {
async function renderFrame(component: () => JSX.Element, options: { width: number; height: number }) {
testSetup = await testRender(component, options)
await testSetup.renderOnce()
await testSetup.renderOnce()
return testSetup
.captureCharFrame()
@@ -294,22 +308,20 @@ describe("TUI inline tool wrapping", () => {
expect(await renderFrame(() => <Fixture before="user" />, { width: 72, height: 14 })).toMatchSnapshot()
})
test("separates a contiguous subagent group from inline tools", async () => {
expect(await renderFrame(() => <SubagentGroupFixture />, { width: 72, height: 10 })).toMatchSnapshot()
test("separates after a multi-line task row", async () => {
expect(await renderFrame(() => <TaskRowsFixture />, { width: 72, height: 10 })).toMatchSnapshot()
})
test("separates a subagent group after an expanded read", async () => {
expect(await renderFrame(() => <LoadedReadBeforeSubagentFixture />, { width: 72, height: 8 })).toMatchSnapshot()
test("does not treat task rows differently from other inline rows", async () => {
expect(await renderFrame(() => <LoadedReadBeforeTaskFixture />, { width: 72, height: 8 })).toMatchSnapshot()
})
test("separates a subagent from the previous assistant summary", async () => {
expect(
await renderFrame(() => <AssistantSummaryBeforeSubagentFixture />, { width: 72, height: 5 }),
).toMatchSnapshot()
test("separates an inline row from the previous assistant summary", async () => {
expect(await renderFrame(() => <AssistantSummaryBeforeInlineFixture />, { width: 72, height: 5 })).toMatchSnapshot()
})
test("separates a subagent from the previous assistant error", async () => {
expect(await renderFrame(() => <AssistantErrorBeforeSubagentFixture />, { width: 72, height: 7 })).toMatchSnapshot()
test("separates an inline row from the previous assistant error", async () => {
expect(await renderFrame(() => <AssistantErrorBeforeInlineFixture />, { width: 72, height: 7 })).toMatchSnapshot()
})
test("updates sticky-bottom geometry when a text separator mounts and unmounts", async () => {