Repository navigation
mcpx(progspec): concurrent resolves of one nix spec each run their own nix build — the memo only dedupes sequential calls #2826
Description
mcpx(progspec): concurrent resolves of one nix spec each run their own nix build — the memo only dedupes sequential calls
Repo: developing-today/code · Main: 1281cb37 · File: pkgs/mcpx/internal/progspec/progspec.go — memo check at lines 136-158, resolveNixMemoized at 168-177 (new in PR #2585, merged 2026-10-30)
Summary
progspec memoises resolved nix specs so "a daemon that restarts a server repeatedly pays the build once" (package doc, lines 22-24). The memo is only consulted before resolving and only populated after a resolution finishes:
memoMu.Lock()
if r, ok := memo[key]; ok { ... return r, nil }
memoMu.Unlock()
return resolveNixMemoized(ctx, spec, dir, key) // resolves, then storesThere is no in-flight tracking, so N concurrent Resolve/ResolveInDir calls for the same spec — the normal shape when a daemon starts or restarts several servers at once, or several servers share one nix: command — each spawn their own nix build --no-link --print-out-paths. Nix's store locking serialises the actual build, but every invocation still pays full flake evaluation (the expensive part for a large flake such as nixpkgs), and each is independently allowed up to DefaultTimeout (3 minutes, line 43).
Evidence (verified this run, real module)
Laptop probe (~/git/fleet/probe-2026-10-30-laneA-progspec/; the repo's real progspec.go, md5 69fd5418546542c4de1f127906d59ed5 identical on sandbox main 1281cb37 and the laptop copy; stub nix on PATH in the style of the package's own tests, sleeping 0.3 s per invocation and counting calls):
| Scenario | nix invocations |
|---|---|
8 goroutines calling Resolve(ctx, "nix:nixpkgs#faketool") concurrently, same spec/dir |
8 (elapsed 312 ms — fully parallel, no dedupe) |
2 sequential resolves of the same spec (existing test TestResolveNixSpecBuildsOnceAndSpawnsTheStorePath) |
1 (memo works sequentially — not in dispute) |
Callers that resolve at spawn time, all reachable concurrently: pkgs/mcpx/internal/mcpclient/stdio.go:103 (ResolvePathInDir), pkgs/mcpx/internal/mcpclient/supervised.go:243 (spawnChild, per child restart) and :609 (ResolvePathFreshInDir).
Why it matters
The memo's stated purpose is paying the build once per daemon. Concurrent first-resolves — daemon startup with several nix: servers, a supervisor restarting children together, pooled servers sharing one spec — silently pay it once per caller instead, multiplying the slowest, most failure-prone step of launch (a network/flake evaluation with a 3-minute timeout) exactly when the daemon is busiest. It also multiplies the failure surface: one transient nix failure can surface as N slightly-different launch errors for what is logically one resolution.
Acceptance criteria
- Concurrent
Resolve/ResolveInDircalls for the same memo key share one in-flightnix build: a regression test with the stub-nixharness (slow stub, N concurrent resolves) counts exactly 1 invocation and all callers receive the sameResolved. - A failed in-flight resolution is not memoised (today's behaviour): every waiter receives that error, and a later call retries the build.
- The deleted-output eviction behaviour (
TestResolveNixSpecMemoEvictsDeletedOutput) and the per-directory memo keying still hold; a resolution whose memo entry is evicted mid-flight does not resurrect a dead path. -
ResolveFreshInDir's contract is preserved or explicitly re-pinned in its doc comment (it bypasses the memo read by design — triage decides whether fresh callers also share an in-flight resolution or stay fully independent). - Existing
pkgs/mcpx/internal/progspectests still pass (go test ./internal/progspec -count=1in CI / on the laptop — notgo test ./...locally, perpkgs/mcpx/internal/AGENTS.md).
Dedup notes: searched open + closed issues this run for progspec / singleflight / concurrent resolve / memo — no hit. Related but distinct: #1921 (the feature this shipped in, still open for its environment half), #1908 (a durable, cross-process content-addressed cache for flake resolutions — this issue is only the in-process concurrent case), #2754/#2726 (mcpx result-validation / settings panics, unrelated paths). NOT self-solved: the singleflight shape (and the ResolveFreshInDir question above) is triage's pin, and fleet verification for Go changes runs on the laptop/CI, not in a creator run.
Activity
dezren39 commentedon Oct 30, 2026
🤖 Agent note · filed
pkgs/mcpx/internal/progspec/progspec.go:136-177(new in PR mcpx: program-spec resolver — nix:<ref>#<attr> resolves once to a store path (#1921 Part 1) #2585) memoises resolved nix specs only between sequential calls — the memo is checked before resolving and populated after, with no in-flight tracking, so concurrent resolves of one spec each run their ownnix build. Verified on the laptop with the real module (md5-matched to main1281cb37): 8 concurrentResolvecalls fornix:nixpkgs#faketoolagainst a 0.3 s stubnixproduced 8 stub invocations (312 ms elapsed, fully parallel), while the sequential control in the package's own tests builds once.ResolveFreshInDirsharing question is pinned. Singleflight shape is triage's pin.82 remaining items
dezren39 commentedon Nov 2, 2026
agent-note:v1 — holder: repo-worker-code; action: picked_up; at: 2026-11-02T12:51:45Z. Queue task 5431 claimed, resuming from verified ownership stop. Fresh metadata confirms #2826 is OPEN and assigned to dezren39; PR #4246 is OPEN, non-draft, authored by and assigned to dezren39. No PR internals, branch, worktree, CI, or reviews inspected.
dezren39 commentedon Nov 2, 2026
agent-note:v1 — holder: repo-worker-code; action: parked; at: 2026-11-02T12:52:10Z. Task 5431 parked for 30 minutes: fresh metadata confirms issue #2826 and open PR #4246 are Drew-owned. No PR contents, branch, worktree, CI, or reviews inspected. Resume only after explicit ownership handoff/release, then revalidate metadata before any work.
dezren39 commentedon Nov 2, 2026
🤖 Agent note · picked up
dezren39 commentedon Nov 2, 2026
🤖 Agent note · parked
dezren39 commentedon Nov 2, 2026
🤖 Agent note · done — PR #4246 merged
queue/5431-2826-progspec-singleflight, this task's own canonical PR; author dezren39 is the shared fleet account. No hard-stop labels were ever present.pkgs/mcpx/internal/progspec: concurrent resolves of one spec share one in-flight nix build; failures are not memoised; cancelled waiters exit without cancelling the leader.