The timestamp approach incorrectly deleted valid baselines that Playwright
didn't rewrite (because they already matched). Instead, read the Storybook
index.json to compute the expected set of screenshot filenames, and only
remove baselines that don't correspond to any current story.
The previous approach of removing all baselines before regeneration caused
spurious changes because Playwright produces slightly different PNGs across
runs (within tolerance but different bytes). Also, skipped stories had their
baselines permanently deleted.
Instead, create a timestamp marker before the test run and delete only
screenshots that weren't touched by Playwright (i.e., stale baselines for
removed stories). This also reverts the spurious baseline changes from the
previous auto-commit.
Add Storybook + Playwright visual regression testing for the VS Code
extension's composite webview UI. This tests the full composition of
kilo-ui components with extension-specific contexts and styles.
- .storybook/ config with kilo-vscode theme defaults
- StoryProviders mock wrapper for DataProvider, SessionContext, etc.
- 7 composite stories: tool+permission, permission dock, tool cards,
chat idle/busy, multi-tool assistant message
- Playwright config on port 6007 with auto-discovery
- CI workflow updated with parallel kilo-vscode visual regression job
- Exported SessionContext and LanguageContext for story use
- Add package.json to Storybook cache key (dep version changes invalidate cache)
- Add res.ok check before parsing Storybook index JSON
- Remove duplicate maxDiffPixelRatio (already in playwright.config.ts)
- Use parseInt for PLAYWRIGHT_WORKERS to handle non-numeric values
The workflow now:
1. Runs --update-snapshots to write new/changed baselines
2. Commits any changes back to the branch
3. Fails the job if baselines changed, so the PR author is notified
This avoids silently accepting visual changes — the author must pull
and review the updated baselines.
- Add @playwright/test, playwright.config.ts (Chromium, 1280x720)
- Test file auto-discovers all stories from Storybook's index.json
- Screenshots cropped to #storybook-root, grouped per component
- CI workflow on Blacksmith with Storybook/Playwright/Bun caching
- Paths filter: only runs when kilo-ui or its deps change
- Auto-commits missing baselines back to the branch via CI
- Skips head-only (Font, Favicon) and non-deterministic (Typewriter) stories
- Documented in CONTRIBUTING.md
Mark IJbema