Skip to content
This repository was archived by the owner on Oct 4, 2026. It is now read-only.

typescript: give declaration compiles a types-only view of npm packages - #23

Merged
slarson-redo merged 4 commits into
mainfrom
slarson/types-only-npm-trees
Sep 30, 2026
Merged

slarson-redo merged 4 commits into
mainfrom
slarson/types-only-npm-trees

Conversation

@slarson-redo

@slarson-redo slarson-redo commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

typescript: give declaration compiles a types-only view of npm packages

Problem

A TypeScriptCompile{,Native} action's inputs include TsInfo.transitive_files of every dependency. For an npm package that is the whole package tree (:files, a copy_directory of src/): JS, maps, docs, tests. Bazel expands trees in the sandbox, so each action symlinks and prefetches every file of every transitive npm package. On redo's type-check chain that is 128-153k input files per action, 84% of them npm, while tsgo opens ~5k.

Change

  • typescript/npm_plugin.bzl: each TS npm spoke also builds :types, a copy_to_directory of :files filtered to **/*.{ts,tsx,mts,cts} and **/*.json (package.json for resolution and package scope, other JSON for resolveJsonModule imports); ts_import(declarations = [":types"], types = ":types"). A package whose only declarations are .d.cts/.d.mts is now treated as a TypeScript package too (previously a redo-local patch).
  • typescript/rules.bzl: ts_import gets a types attr (it requires root, and fails without one). The package struct gets types_path (create_package), TsInfo is built without the root's descriptors (for npm they are the full :files tree, which would otherwise stay an input), and the compile package manifest uses types_package_path (types_path or path). The manifest already separates id from path, so the dependency graph is untouched; stage-nm and the tsconfig paths resolve into the smaller tree.
  • TsCompilerInfo.runtime_ts: the compiler runtime (tslib) is CjsInfo+JsInfo only, so its types tree would be missing; its TsInfo is added to compile inputs and TsCompileInfo.declarations (lint).
  • Runtime paths (JsInfo, CjsInfo, nodejs/jest/webpack) are unchanged.

Results (redo monorepo)

Input files per declaration action (aquery + tree counts, verified exact against materialised trees):

action before after
agents/platform-agents:lib 145,633 69,171 (-52%)
merchant/server:lib 153,421 76,461 (-50%)
order-editing/service:lib 128,591 61,189 (-52%)
orders/service:lib 65,330 32,854 (-50%)

All 10,045 native compile actions: p50 30.5k -> 15.4k, p90 104k -> 51k, aggregate -50.5%.

CI-shaped A/B (real MR chain, CI flags, --jobs=28, fresh output base, warm disk-cache seed, 5 reps each, Coder 16 CPU, processwrapper-sandbox), critical-path declaration actions:

before after
CP declaration time 728.9 s 620.9 s (-15%)
input prefetch 91.8 s 39.0 s (-58%)
sandbox setup (createFileSystem) 85.3 s 45.2 s (-47%)
compiler 354.2 s 347.0 s (-2%)
Bazel wall (median) 858 s 787 s (-8%)

Correctness: //:dts plus scoped and 14 extra tslint targets (tsgo and node tsc) pass with 0 errors, and all 61,633 emitted .d.ts/.d.cts/.d.mts are byte-identical (sha256) to the previous build.

Known difference

A JS-only subpath of a package that also ships some .d.ts no longer resolves to an untyped JS file; it is "not found". Both are errors under strict settings and none exists in redo. allowJs on a ts_library's own JS sources still works (covered by a test): declaration compiles never analyze npm JS (maxNodeModuleJsDepth 0), so the only effect is that JS-only npm subpaths are not found.

Same cause, seen in rules_javascript's own tests: rxjs 7.4.0 has no types exports condition, so under bundler resolution it only "resolved" by finding its JS (implicitly any). It is now not found, so the test fixture moves to rxjs 7.8 (declares types).

Also fixed

  • bazel_lib 3.2.1's copy_to_directory opts into path mapping but writes unmapped paths into its config JSON, so :types failed under --experimental_output_paths=strip (rules_javascript's preset; redo runs with it off). bazel_lib 3.5.0 (what redo pins) drops path mapping for that action; rules_javascript now requires it.
  • angular_library compiles use the types-only package paths, like ts_library.

Cost

Keeping all JSON (not just package.json) adds ~1.3% to the types trees (284 of 21,279 files over 721 trees in orders/service + s3-client + react/animation), so the input counts above rise by roughly that. One extra copy_to_directory action per npm package a compile touches (+5.7% actions). Cacheable; cold cost was ~3.7% of action time once. Declaration compile cache keys all change on adoption (one full rebuild).

slarson-redo and others added 4 commits September 29, 2026 22:37
An npm package's TsInfo carried the whole package tree (JS, maps, docs,
tests), which the sandbox expands to one symlink and one prefetched file
per action. Declaration compiles read only .ts sources and package.json.

Each npm spoke now also builds a `types` tree (copy_to_directory of
:files filtered to **/*.{ts,tsx,mts,cts} and **/package.json) and
ts_import takes it via a new `types` attr:

- the package struct gets types_path (create_package), and the compile's
  package manifest uses it instead of path, so stage-nm and tsconfig
  paths resolve into the smaller tree (manifest id is unchanged);
- TsInfo is built without the root's descriptors, which for npm are the
  full :files tree;
- the compiler runtime's TsInfo is added to compile inputs
  (TsCompilerInfo.runtime_ts) so tslib's types tree is present.

Known difference: a JS-only subpath of a package that also ships some
.d.ts now resolves to "not found" instead of an untyped JS file.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…aration compiles

- :types keeps **/*.json, so resolveJsonModule imports of npm JSON resolve.
- Declaration tsconfigs set allowJs: false. The types-only trees hold no
  JavaScript, so a config relying on allowJs now errors (TS6504 for JS
  sources, TS5052 for checkJs) instead of resolving less than it claims.
- ts_import fails when types is set without root.
- Drop include_external_repositories: copy_to_directory already keeps files
  from its own repository.
- Treat .d.cts/.d.mts-only packages as TypeScript packages (from redos
- Bump bazel_lib to 3.5.0 (what redo pins). 3.2.1 copy_to_directory opts into
  path mapping but writes unmapped paths into its config JSON, so the :types
  action lstat-ed a path that does not exist in a path-mapped (strip) build.
  Later versions drop path mapping for that action.
- angular_library compiles with the types-only package paths, matching
  ts_library; its inputs no longer contain the full package trees.
- Test fixture: rxjs 7.4.0 has no "types" exports condition, so under bundler
  resolution it only resolved by finding the JS file (implicitly any). With
  JS out of the compile it is correctly not found; use rxjs 7.8, which
  declares types.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The pin broke a ts_library that compiles its own JS sources with allowJs
(redo's uptick shopify-extensibility: TS6504 on .js/.jsx). The pin only
guarded against npm JS, which a declaration compile never analyzes anyway
(maxNodeModuleJsDepth is 0), so it rejected valid first-party configs for
no benefit. Adds a regression test compiling a .js source with allowJs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@slarson-redo
slarson-redo merged commit f940c4f into main Sep 30, 2026
2 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant