8aaa62c794
* feat(session-export): scaffold module with config constants * feat(session-export): add zstd compression wrapper * feat(session-export): event and envelope type definitions * feat(session-export): eligibility check with kill-switch * feat(session-export): org signal collector with auth resolver * feat(session-export): worker SQLite schema and storage helpers * feat(session-export): content-addressed chunker with zstd dedup * feat(session-export): client-side light scrubber * feat(session-export): IPC contract and inbox with back-pressure * feat(session-export): persist scrubbed events with chunked payloads * feat(session-export): worker entry point with inbox drain loop * feat(session-export): main-thread capture module * feat(session-export): workspace baseline and delta fibers * feat(session-export): sync subscriber and tool io chunking * feat(session-export): bootstrap wiring and compaction hook * feat(session-export): wire capture hooks into sessions * feat(session-export): uploader and buffer cap * chore(session-export): annotate shared session hooks * changeset(session-export): add release note * fix(session-export): keep bootstrap non-blocking without instance context * feat(session-export): respawn worker after failures * test(session-export): add performance budget assertions * test(session-export): add worker end-to-end smoke test * fix(session-export): strip identity and high-risk baseline paths * test(session-export): gate perf assertions for stable sweeps * fix(session-export): bundle worker in single-file builds * fix(session-export): make capture envelopes cloneable * fix(session-export): drain worker on CLI shutdown * feat(session-export): capture workspace baseline and deltas * fix(session-export): preserve request metadata * test(session-export): cover request metadata capture * feat(cli): send indexed session export batches * feat(cli): authorize session export uploads * fix(cli): flush session export shutdown uploads * feat(cli): optimize session export replay payloads * fix(cli): include session export surface metadata * fix(session-export): decrement chunk refs after upload decRefChunks was never called, so chunk ref_count stayed at 1 (or higher with dedup) and DELETE FROM chunk WHERE ref_count <= 0 never matched. Chunks accumulated in the local SQLite buffer until the 50 GB cap. Call decRefChunks alongside markUploaded so deleteUploaded can reclaim the rows. * fix(session-export): add periodic uploader flush timer flushIntervalMs and retryBackoffMaxMs were both defined in config and neither was referenced anywhere; scheduleFlush only ran on inbound events or reconnect. After a 5xx or network failure the row was backed off for 1 s, but if no further event arrived the retry never fired and the events stranded until the CLI restarted. Drive a periodic flush from a setInterval, unref the handle so it doesn't pin the process, and clear it from a new dispose() hook the worker calls on shutdown. * fix(session-export): stop emitting absolute workspace root in baseline CaptureMetadata.root was the literal absolute filesystem path (/Users/<name>/Projects/<repo>), shipped in every workspace_baseline_completed event and not stripped by handlers' identity filter. The field was set but never read anywhere downstream — file paths in the baseline are already relative, so the root added no replay signal. Remove the field outright. * fix(session-export): cap pendingEvents result set The SELECT had no LIMIT clause, so under a backlog (network outage, crashed receiver) it would materialize the full pending table into a JavaScript array before the byte-limit truncation applied. With a 50 GB buffer cap that is hundreds of MB of heap inside the worker per drain. Add LIMIT 500 — the drain loop already re-queries until empty, so no events are missed. * fix(session-export): exponential retry backoff up to retryBackoffMaxMs Both the 5xx and network-error branches always retried after the floor delay regardless of how many attempts had already failed, and retryBackoffMaxMs was unreferenced. During a sustained outage every session re-tried at roughly 1 Hz against the dead receiver. Surface upload_attempts on EventRow and compute the next delay as min * 2^attempts capped at max. * fix(session-export): evict superseded workspace snapshots on remember createWorkspaceProvider retained every captured snapshot — in-memory and inside the persisted state file — even after the session moved on to a newer one. For a 1k-file repo over a 100-turn session that is ~2 GB of unreachable heap plus a state file that grows monotonically. Drop the previous snapshot for the session on remember() when no other session still references it. * fix(session-export): anchor aws_secret_key scrubber to key name The bare 40-char base64 pattern matched every 40-character hex string, including all git commit SHAs. Tool outputs, diffs, and conversation messages were silently rewritten as <<REDACTED:aws_secret_key>>, destroying lineage information in training data. Require the key name context — naked secrets in unstructured text are rare and the .env / .aws/credentials high-risk path strip already covers the common case. * fix(session-export): preserve root linkage in SyncSubscriber events SyncSubscriber hardcoded rootSessionId = sessionId on every tool, permission, and feedback event, so sub-agent sessions lost their root linkage and a future training pipeline could not reconstruct the agent topology from these side-channel events. Expose the rootSessionId mapping from Capture and plumb it through the same pattern as getTurnId. * fix(session-export): atomic chunk GC after upload markUploaded + decRefChunks + deleteUploaded were three separate SQL statements; a crash between markUploaded and decRefChunks would leave events flagged uploaded (never retried) and chunks with stale ref_count (never reclaimed by deleteUploaded). Bundle the three calls into a single transactional commitUploaded helper so either all three land or none of them do. * fix(session-export): wait on transient sqlite locks * fix(session-export): snapshot current workspace directory * fix(cli): preserve Kilo model export metadata * fix(cli): send anon id for session export * fix(cli): fallback to telemetry anon id * chore(cli): annotate session export config test * chore: remove session export docs * fix(cli): harden session export uploads * fix(cli): preserve stream lifecycle for session export * fix(kilo-docs): exclude session export ingest link * fix(cli): restrict session export workspace sync to git repos * chore: remove session export changeset * fix(cli): tighten session export payload types * fix(cli): type session export model payloads * fix(cli): simplify session export cleanup * fix(cli): avoid session export shutdown race * refactor(cli): use drizzle for session export storage * fix(cli): finalize session export sqlite statements * fix(cli): link compaction exports to root sessions * fix(cli): stop exporting raw stream parts * fix(cli): prune stale workspace snapshots * refactor(cli): clarify chunk ref counting * test(cli): clarify dropped upload assertions * ci: avoid visual path filter action failure * fix(cli): preserve in-flight workspace snapshots * fix(cli): stop uploading baseline start events * fix(cli): trim redundant export metadata * fix(cli): trim workspace export bookkeeping * fix(cli): fold terminal outcome into tool exports * fix(cli): dedupe request context in export batches * fix(cli): normalize compaction export payloads * fix(cli): avoid duplicate tool result exports * test(cli): align session export expectations * fix(cli): run secretlint during session export scrubbing * fix(cli): keep retried export batches contiguous * fix(cli): include agent info in session exports * fix(cli): flush pending session exports on serve startup * fix(cli): drop exports when scrubbing fails * fix(cli): narrow secretlint value extraction * fix(cli): limit exported agent info * test(cli): remove brittle agent export source assertion * fix(cli): ignore corrupt session export workspace state * fix(cli): keep session export close best effort * fix(cli): avoid following session export symlinks * fix(cli): preserve session export chunk ref counts * fix(cli): retry transient session export uploads * fix(cli): validate session export ingest endpoint * fix(cli): tolerate missing export token details * fix(cli): fail closed on session export org lookup * fix(cli): decode session export permission replies * fix(cli): finalize session export on stream close * fix(cli): bound session export baseline timeout * fix(cli): avoid persisting workspace file contents * fix(cli): bound workspace snapshot capture * fix(cli): validate session export worker messages * fix(cli): revoke stale session export eligibility * fix(cli): scope session export workspaces * fix(cli): extend session export shutdown flush * fix(cli): infer free Kilo models for export * fix(cli): avoid duplicate chunk refs * fix(cli): throttle session export uploads * fix(cli): harden session export capture * feat: disclose free model data collection (#10767) * feat: disclose free model data collection * chore(cli): document free model footer sorting * fix(vscode): add free model data translations * fix: simplify free model data label * fix: simplify data collection badges * fix: remove duplicate model info disclosure * fix: restore composer data tooltip * fix: align jetbrains data collection indicator * fix: align free model data indicators * fix(vscode): show data collection in model preview * fix: limit data indicators to kilo gateway * test(cli): relax prompt cancel timeout * test(cli): annotate prompt cancel timeout * test(cli): classify prompt queue runtime test * fix(cli): defer session export startup
126 lines
6.5 KiB
TypeScript
126 lines
6.5 KiB
TypeScript
#!/usr/bin/env bun
|
|
// kilocode_change - new file
|
|
|
|
/**
|
|
* Prevents new service-local runtimes in shared Effect modules while the
|
|
* remaining Kilo Promise facades are migrated away. It also prevents tests
|
|
* from reaching through the global application runtime unless the integration
|
|
* boundary is explicitly classified.
|
|
*
|
|
* Existing sites are allowed only when classified below. Remove transitional
|
|
* entries after their migration lands so later reintroductions fail CI.
|
|
*/
|
|
|
|
import path from "node:path"
|
|
|
|
const ROOT = path.resolve(import.meta.dir, "..")
|
|
const DIR = path.join(ROOT, "packages", "opencode", "src")
|
|
const TEST_DIR = path.join(ROOT, "packages", "opencode", "test")
|
|
const PATTERN = /makeRuntime\s*\(\s*Service\s*,/g
|
|
const TEST_PATTERN = /\bAppRuntime\b/g
|
|
|
|
const allow: Record<string, string> = {
|
|
"bus/index.ts": "core bus callback and synchronous runtime boundary",
|
|
"cli/cmd/run/runtime.boot.ts": "direct run startup resolver runtime boundary",
|
|
"cli/cmd/run/stream.transport.ts": "per-subscription direct run transport runtime boundary",
|
|
"cli/cmd/run/variant.shared.ts": "direct run variant persistence runtime boundary with test filesystem injection",
|
|
"cli/cmd/tui/config/tui.ts": "separately tracked TUI config facade",
|
|
"installation/index.ts": "existing installation facade outside #10655",
|
|
"session/compaction.ts": "existing compaction facade outside #10655",
|
|
"sync/index.ts": "sync event runtime boundary",
|
|
}
|
|
|
|
const testAllow: Record<string, { count: number; reason: string }> = {
|
|
"config/agent-color.test.ts": { count: 2, reason: "existing runtime integration test" },
|
|
"config/tui.test.ts": { count: 3, reason: "existing runtime integration test" },
|
|
"control-plane/workspace.test.ts": { count: 11, reason: "existing runtime integration test" },
|
|
"effect/app-runtime-logger.test.ts": { count: 6, reason: "tests AppRuntime behavior" },
|
|
"kilocode/config-resilience.test.ts": { count: 4, reason: "existing runtime integration test" },
|
|
"kilocode/config-validation.test.ts": { count: 2, reason: "existing runtime integration test" },
|
|
"kilocode/plan-followup.test.ts": { count: 7, reason: "existing runtime integration test" },
|
|
"kilocode/server/config-overlay.test.ts": { count: 3, reason: "server config cache integration test" },
|
|
"kilocode/session/platform-attribution.test.ts": { count: 5, reason: "existing runtime integration test" },
|
|
"kilocode/session-prompt-queue.test.ts": { count: 5, reason: "prompt queue legacy instance bridge regression" },
|
|
"kilocode/session/session.test.ts": { count: 4, reason: "existing runtime integration test" },
|
|
"mcp/headers.test.ts": { count: 4, reason: "existing runtime integration test" },
|
|
"mcp/oauth-browser.test.ts": { count: 4, reason: "existing runtime integration test" },
|
|
"permission-task.test.ts": { count: 2, reason: "existing runtime integration test" },
|
|
"project/vcs.test.ts": { count: 14, reason: "existing runtime integration test" },
|
|
"provider/amazon-bedrock.test.ts": { count: 2, reason: "existing runtime integration test" },
|
|
"provider/provider.test.ts": { count: 3, reason: "existing runtime integration test" },
|
|
"pty/pty-output-isolation.test.ts": { count: 4, reason: "existing runtime integration test" },
|
|
"pty/pty-session.test.ts": { count: 3, reason: "existing runtime integration test" },
|
|
"pty/pty-shell.test.ts": { count: 4, reason: "existing runtime integration test" },
|
|
"session/llm.test.ts": { count: 2, reason: "existing runtime integration test" },
|
|
"tool/recall.test.ts": { count: 10, reason: "existing runtime integration test" },
|
|
}
|
|
|
|
const owned = (file: string) => file.startsWith("kilocode/") || file.startsWith("kilo-sessions/")
|
|
const hits: Array<{ file: string; line: number }> = []
|
|
const glob = new Bun.Glob("**/*.ts")
|
|
|
|
for (const file of glob.scanSync({ cwd: DIR, onlyFiles: true })) {
|
|
if (owned(file)) continue
|
|
const text = await Bun.file(path.join(DIR, file)).text()
|
|
for (const match of text.matchAll(PATTERN)) {
|
|
const line = text.slice(0, match.index ?? 0).split("\n").length
|
|
hits.push({ file, line })
|
|
}
|
|
}
|
|
|
|
const invalid = hits.filter((hit) => !allow[hit.file])
|
|
const drift = Object.entries(allow).flatMap(([file, reason]) => {
|
|
const count = hits.filter((hit) => hit.file === file).length
|
|
if (count === 1) return []
|
|
return [` packages/opencode/src/${file}: expected 1 classified site, found ${count} (${reason})`]
|
|
})
|
|
|
|
const testHits: Array<{ file: string; line: number }> = []
|
|
for (const file of glob.scanSync({ cwd: TEST_DIR, onlyFiles: true })) {
|
|
const text = await Bun.file(path.join(TEST_DIR, file)).text()
|
|
for (const match of text.matchAll(TEST_PATTERN)) {
|
|
const line = text.slice(0, match.index ?? 0).split("\n").length
|
|
testHits.push({ file, line })
|
|
}
|
|
}
|
|
|
|
const testInvalid = testHits.filter((hit) => !testAllow[hit.file])
|
|
const testDrift = Object.entries(testAllow).flatMap(([file, entry]) => {
|
|
const count = testHits.filter((hit) => hit.file === file).length
|
|
if (count === entry.count) return []
|
|
return [
|
|
` packages/opencode/test/${file}: expected ${entry.count} classified reference(s), found ${count} (${entry.reason})`,
|
|
]
|
|
})
|
|
|
|
if (invalid.length > 0 || drift.length > 0 || testInvalid.length > 0 || testDrift.length > 0) {
|
|
if (invalid.length > 0) {
|
|
console.error("Found unclassified service-local Effect runtimes in shared opencode modules:")
|
|
for (const hit of invalid) console.error(` packages/opencode/src/${hit.file}:${hit.line}`)
|
|
console.error("")
|
|
}
|
|
if (drift.length > 0) {
|
|
console.error("Classified service-local runtime exceptions no longer match the current source:")
|
|
for (const item of drift) console.error(item)
|
|
console.error("")
|
|
}
|
|
if (testInvalid.length > 0) {
|
|
console.error("Found unclassified AppRuntime use in opencode tests:")
|
|
for (const hit of testInvalid) console.error(` packages/opencode/test/${hit.file}:${hit.line}`)
|
|
console.error("")
|
|
}
|
|
if (testDrift.length > 0) {
|
|
console.error("Classified test AppRuntime exceptions no longer match the current source:")
|
|
for (const item of testDrift) console.error(item)
|
|
console.error("")
|
|
}
|
|
console.error("Do not add Promise facades to shared Effect services or global AppRuntime dependencies to tests.")
|
|
console.error("Yield services directly in scoped layers, or classify intentional integration boundaries explicitly.")
|
|
console.error("Remove migrated exceptions, or classify intentional runtime changes with an explicit reason.")
|
|
process.exit(1)
|
|
}
|
|
|
|
console.log(
|
|
`check-opencode-promise-facades: ${hits.length} classified runtime site(s), ${testHits.length} classified test reference(s), no runtime drift found.`,
|
|
)
|