Skip to content

fix(tui): close SQLite stores deterministically so Windows can delete t.TempDir - #4210

Merged
dgageot merged 2 commits into
mainfrom
fix/windows-sqlite-tempdir-teardown
Sep 9, 2026
Merged

dgageot merged 2 commits into
mainfrom
fix/windows-sqlite-tempdir-teardown

Conversation

@aheritier

@aheritier aheritier commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Automated architect agent — this comment was posted by the architect bot from Docker Agentic Platform, not by a human engineer

Problem

windows-tests flakes (~6 % of ci runs over the last 100) with, in 3 of the 6 Windows-only failures:

--- FAIL: TestBackgroundAgent_PerAgentContextInSidebar (2.65s)
    testing.go:1617: TempDir RemoveAll cleanup: unlinkat C:\...\Temp\...\data\tui_state.db:
    The process cannot access the file because it is being used by another process.

(also session.db in TestChat_DegenerateResizeDoesNotPanic). Runs: 34329154242, 34194019096, 34325172091.

Windows refuses to delete a file with an open handle; Go's t.TempDir() cleanup retries sharing violations for only 2 s. Two SQLite handles in the tuitest-driven e2e/tui tests outlived that window:

  • tui_state.db is closed only by the detached goroutine armed in contextShutdownCmd (<-ctx.Done(); cleanupManagedResources()), which nothing joins. tuitest's cleanup quits the program but never releases the model's resources, so a WAL checkpoint + fsync on a slow runner races t.TempDir. In-package unit tests already work around this with t.Cleanup(m.cleanupManagedResources); the external e2e/tui package couldn't.
  • session.db: sql.DB.Close() only closes idle connections. One checked out by an in-flight persistence write (token usage / title, on the runtime observer goroutine) is released after Close returned — the Failed to persist token usage … sql: database is closed warnings in the logs are the same writers landing a moment later.

Fix

  • sqliteutil.CloseDB(db): Close() then wait (bounded, 5 s) until db.Stats().OpenConnections == 0. Used by tuistate.Store.Close. session.SQLiteSessionStore.Close gets the same database/sql-only drain inline, because pkg/session must stay free of the SQLite driver (e2e/dependencies_test.go; see pkg/session/sqlitestore).
  • appModel.Shutdown() (exported, wraps the sync.Once-guarded cleanupManagedResources); tuitest.New calls it right after the program stops when the model implements it — same optional-interface pattern as SetProgram.

Result: every driver-based test closes tui_state.db synchronously before rt.Close, store.Close and t.TempDir run, and store closes don't return until the file handle is gone.

Validation

  • go build ./..., golangci-lint run on touched packages: clean
  • go test ./pkg/sqliteutil/... ./pkg/session/... ./pkg/tui/... ./e2e/tui/... ./pkg/app/... ./pkg/runtime/ ./cmd/root/: pass (Linux)
  • New tests: TestCloseDB_WaitsForInFlightConnection, TestCloseDB_IdleReturnsImmediately, TestSQLiteSessionStore_CloseWaitsForInFlightConnection, TestDriver_CleanupShutsDownModelAfterProgramStops
  • Windows: cannot be exercised locally; the windows-tests job on this PR is the check.

Not in this PR

The other 3 Windows-only flakes are timing budgets (PowerShell start-up under load: pkg/hooks TestExecuteStop, backgroundjobs recall test; pkg/userconfig 5 s lock timeout falling back to unlocked writes). Diagnosis and a prioritized plan for those are in the linked task report.

… t.TempDir

The windows-tests CI job flakes with:

    TempDir RemoveAll cleanup: unlinkat ...\data\tui_state.db (or session.db):
    The process cannot access the file because it is being used by another process.

Windows refuses to delete a file while a handle is open, and Go's testing
package retries for only 2s. Two things kept SQLite handles alive past
that window in the tuitest-driven e2e/tui tests:

- tui_state.db was closed only by the detached goroutine armed in
  contextShutdownCmd, gated on ctx.Done() and never joined. tuitest's
  cleanup quit the program but never released the model's resources, so
  the close (WAL checkpoint + fsync on a slow runner) raced t.TempDir.
- sql.DB.Close only closes idle connections; one checked out by an
  in-flight persistence write (token usage, title) is released after
  Close has already returned, unobserved by the test.

Add sqliteutil.CloseDB, which closes and then waits (bounded) for every
connection to be released, and use it from the tuistate and session
stores. Export Shutdown on the TUI model and have tuitest.New call it
after the program stops, so every driver-based test closes tui_state.db
synchronously before rt.Close, store.Close and t.TempDir run.
@aheritier
aheritier requested a review from a team as a code owner September 9, 2026 10:11
Importing pkg/sqliteutil from pkg/session linked modernc.org/sqlite into
the lean embedder surface guarded by e2e/dependencies_test.go (and broke
the GOOS=js build). Give SQLiteSessionStore.Close its own database/sql-only
drain instead of calling sqliteutil.CloseDB.
@aheritier aheritier added area/sessions For features/issues/fixes related to session lifecycle (resume, persistence, export) area/testing Test infrastructure, CI/CD, test runners, evaluation area/tui For features/issues/fixes related to the TUI kind/fix PR fixes a bug (maps to fix:). Use on PRs only. labels Sep 9, 2026
@dgageot
dgageot merged commit cfcca04 into main Sep 9, 2026
13 checks passed
@dgageot
dgageot deleted the fix/windows-sqlite-tempdir-teardown branch September 9, 2026 14:11
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
docker-agent 1.137.0

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre>This release removes the `session_plan` toolset, adds audio/video/image input and output capability handling, expands hook functionality with new builtins and sequential pipelines, and includes several bug fixes and safety improvements.

## Breaking Changes
- Removes the `session_plan` toolset, including its tool handlers, stream event, plans service, TUI `/plans` browser, and `--session`/`--scope` addressing on the `plans` command

## What's New
- Adds detection and filtering of audio/video input modalities per request model, stripping unsupported media parts and preserving provider-specific fallbacks
- Adds `output_capabilities.image` model override to resolve image output capability from models.dev metadata
- Adds guard for image-output requests across Google (gateway, direct Gemini API, and Vertex) surfaces, rejecting incompatible custom tools and structured output before dispatch
- Adds `add_context` builtin hook for dependency-free Go template-based context injection into the model's conversation
- Adds sequential pipeline execution for `pre_tool_use`, `before_llm_call`, and `tool_response_transform` hooks, replacing the previous concurrent first-rewrite-wins strategy
- Adds 56 new safe and 27 new destructive shell safety patterns, covering Git read operations, GitHub CLI queries, Go tooling, `rg`/`ripgrep`, and common inspection commands

## Bug Fixes
- Fixes Gemini keepalive SSE events (`event: keepalive` with `data: {}`) being passed to the SDK parser, causing parse failures; these frames are now dropped at the transport layer
- Fixes image-output-capable models being incorrectly excluded from session title generation
- Fixes shell metacharacter detection and corrects `gh`/`rg` pattern classifications
- Fixes SQLite stores not being closed deterministically on Windows, causing `TempDir` cleanup failures in tests
- Fixes `pkg/session` importing the SQLite driver, keeping the package free of that dependency

## Technical Changes
- Adds request-shape diagnostics for Gemini image requests (excluding prompt, schema, media-payload, and credential fields)
- Adds shared UTF-8-safe display-name sanitization helpers
- Classifies Gemini API 400 errors into bounded actionable categories
- Adds documentation tip for injecting session ID into model context using a `session_start` hook
---

## What's Changed
* docs: update CHANGELOG.md for v1.136.0 by @docker-read-write[bot] in docker/docker-agent#4202
* fix: drop Gemini keepalive SSE events before SDK parsing by @dgageot in docker/docker-agent#4203
* feat(plan)!: remove the session_plan toolset by @trungutt in docker/docker-agent#4199
* docs: add tip for injecting session ID with a session_start hook by @dgageot in docker/docker-agent#4205
* feat(hooks): add add_context builtin for dependency-free template context injection by @dgageot in docker/docker-agent#4206
* feat(hooks): sequential pipeline for pre_tool_use, before_llm_call, and tool_response_transform by @dgageot in docker/docker-agent#4207
* feat(#3996): resolve and filter input media per request model by @aheritier in docker/docker-agent#4016
* fix(#3996): diagnose Gemini requests and sanitize API failures by @aheritier in docker/docker-agent#4017
* feat(#3996): resolve image output capability from models.dev by @aheritier in docker/docker-agent#4019
* feat(#3996): guard image-output requests across Google surfaces by @aheritier in docker/docker-agent#4020
* fix(#3996): keep image-output models eligible for session titles by @aheritier in docker/docker-agent#4021
* fix(#3996): filter gateway SSE keepalives and test image requests by @aheritier in docker/docker-agent#4022
* feat(safety): expand shell safety patterns and harden substitution checks by @dgageot in docker/docker-agent#4209
* fix(tui): close SQLite stores deterministically so Windows can delete t.TempDir by @aheritier in docker/docker-agent#4210


**Full Changelog**: docker/docker-agent@v1.136.0...v1.137.0
</pre>
  <p>View the full release notes at <a href="https://github.com/docker/docker-agent/releases/tag/v1.137.0">https://github.com/docker/docker-agent/releases/tag/v1.137.0</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!18818
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/sessions For features/issues/fixes related to session lifecycle (resume, persistence, export) area/testing Test infrastructure, CI/CD, test runners, evaluation area/tui For features/issues/fixes related to the TUI kind/fix PR fixes a bug (maps to fix:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants