Negative input pattern is ignored for files discovered via readdir #504

Activity

1. fengmk2

   assigned

   wan9chi

   on Jul 3, 2026
2. fengmk2 commented on Jul 3, 2026

   

   fengmk2

   on Jul 3, 2026

   MemberAuthor

   More actions

   Related: #505 — a separate caching issue found in the same Vite + void GitHub Actions setup (a broad tracked env query validated against the unfiltered parent env).
3. fengmk2

   added a commit that references this issue on Jul 3, 2026

   Cache the CI build with vite-task (#18)

   ...

   8673e5b
4. jasonkuhrt commented on Sep 9, 2026

   

   jasonkuhrt

   on Sep 9, 2026

   More actions

   Adding a data point from a large monorepo (Heartbeat, ~8,800 tracked files, ~50 packages), plus where the fix appears to sit.

   Effect at scale

   Nearly every task in our workspace lists the workspace root: Vitest project discovery, pnpm-workspace.yaml globbing, tools resolving the workspace, and so on all readdir the root, so the root listing (66 entries for us) is a fingerprinted input of almost every task. Because negative patterns are not consulted for listing children, adding or removing any top-level file or directory misses the cache for every root task at once, whatever the negatives say. We measured 4 such commits in our last 300 on trunk; each one reran roughly 14 minutes of otherwise cacheable CI.

   The reproduction in the issue is exactly what we see. The message on a miss is '<name>' added in workspace root, even when the task declares !<name> and !<name>/**.

   Where the two behaviours diverge

   Reading the tree at 6844d20:

   • Negatives are applied to path reads: session/execute/mod.rs builds input_negative_globs as a PathGlobSet from input_config.negative_globs and filters tracked reads with it. That is why the direct-path case in the repro hits.
   • Listing children never see that set: session/execute/fingerprint.rs stores each listed folder as a BTreeMap of entry names, and determine_folder_change_kind reports the first added or removed name by walking the old and new maps in lockstep. Nothing between the raw listing and that comparison filters names, apart from the fixed should_ignore_entry list (., .., .DS_Store, dist).

   So a listed directory's fingerprint carries every child name, and the negatives that would have excluded a child as a file read cannot exclude it as a listing entry.

   Proposed shape of a fix

   Filter listing children with the same input_negative_globs at fingerprint time, in both places that build a folder fingerprint: when the post-run fingerprint is recorded, and in validate when the current listing is re-read for comparison. A child whose full relative path (<dir>/<name>) matches a negative pattern would be dropped from the map before it is stored or compared. That keeps the current semantics for everything else and makes the repro's readdir-case behave like direct-path-case.

   Happy to send a PR along those lines if that direction is acceptable.
5. jasonkuhrt commented on Sep 21, 2026

   

   jasonkuhrt

   on Sep 21, 2026

   More actions

   Two follow-ups from living with this for a while. The first is a smaller fix than the one I proposed above; the second is a diagnostic problem that costs real time.

   A fix that needs no configuration at all

   My earlier suggestion — filter listing entries with input_negative_globs — still requires every task to spell its negatives correctly, and the common case does not need configuration to be decidable.

   The dominant case is a task that writes into a directory it also lists. The fingerprint is taken after the run, so the listing always contains what the task just produced, and a fresh checkout always differs. The engine already knows which paths those are: observe_fspy collects path_writes in the same pass as path_reads (session/execute/cache_update.rs).

   So: when building a Folder(Some(entries)) fingerprint, drop entries the task itself created. No negative patterns, no config, and it is strictly more accurate than a glob — a path the task wrote is not an input by definition.

   The two fixes compose. Self-written entries are handled automatically; declared negatives still cover entries a different process owns, like another package's generated output or a sibling tool's scratch.

   The miss message is ambiguous, and it points at the wrong file

   'X' removed from 'Y' has two distinct origins that render identically:

   1. the key Y/X was an input and is now gone — determine_change_kind returns entry_name: None, path is the key;
   2. the key Y is a listing that no longer contains entry X — entry_name: Some("X"), path is synthesised as Y/X.

   Only (2) is this bug. They are indistinguishable in the log, and the distinction decides the fix: (1) is cured by a negative naming Y/X, (2) only by one that reaches Y.

   This is not hypothetical. Today I read 'next-env.d.ts' removed from 'apps/public-pages', concluded it was the package directory's listing, excluded the directory — and it still missed, because it was case (1): Next reads the file back. The exclusion I needed was for the file, and the one I wrote dropped real invalidation for nothing.

   Distinguishing them in the message would be enough. Something like 'X' removed from listing of 'Y' for (2), or naming the recorded input path for (1), so it is clear which path the negative has to match.

   For reference, the shape we hit most often, all against vite-plus 0.3.3:

   cache miss: 'next-env.d.ts' removed from 'apps/public-pages'      # (1) file read back
   cache miss: 'types' removed from 'apps/public-pages/.next'        # (2) listing of .next
   cache miss: 'assets' removed from 'parts/backend-services'        # (2) listing of the package dir
   cache miss: '__generated_source__' removed from 'parts/central-icon/src'   # (2), and the writer is another package
   

   Eleven failures of our cache-determinism job over the last hundred runs are this one mechanism. Happy to send the path_writes patch if that direction is acceptable.