[전수조사/L] 테스트·CI·보안·성능 #683

테스트 격리, CI 보안, 성능 측정, 미지수를 확인할 테스트 인프라입니다.

전체 계획과 결정 사항은 추적 이슈(#682)에 있습니다. 조사 기준 커밋: 77daad74 (현재 main d0b84255).

결정에 따른 작업

• (필수) #[serial] 누락(INF-01, KNOWN-16): 전역 theme 상태를 쓰는 테스트
• (권장) 미지수 해소 테스트 인프라: Firefox·WebKit, 실제 번들러 end-to-end(next dev/build, vite, rsbuild), React 18 fixture

발견 항목

심각도: P0 = 흔한 사용에서 조용히 틀리거나 크래시, P1 = 흔한 패턴이 명시적 에러로 막히거나 드물게 조용히 틀림, P2 = 드문 경우·도구/문서 불일치, P3 = 있으면 좋음.

필수 (1)

• INF-01 (P1, NONDET) TOKEN_REGISTRY has multiple non-serial writers
  • 근거: theme_tokens.rs:82-119 has three plain #[test] writers; sheet::test_get_theme_interface (2663+) and test_theme_layer_in_css (3025+) also call set_theme; serial tests such as test_conditional_typography_pipeline depend on the same registry.
  • 수정 방향: mark every registry reader/writer serial or replace process-global registration with scoped/per-sheet state and a reset guard.

권장 (7)

• INF-03 (P1, NONDET) Bun files share persistent WASM and plugin process state
  • 근거: root bunfig.toml runs package tests together; next-plugin/src/wasm.ts:10-61 caches modules; wasm.test.ts:39-112 resets overrides/CWD but not caches or Rust global state.
  • 수정 방향: expose/use a test-only aggregate reset, avoid process CWD mutation, and serialize stateful files.
• INF-04 (P1, TEST) Rust 100% excludes the actual JS/WASM ABI
  • 근거: all public extraction/map/theme/resolver/query wrappers at binding lines 51,226-329,468-671 are cfg(not(tarpaulin_include)); wasm32 RefCell paths are also excluded.
  • 수정 방향: add wasm-bindgen/Node integration coverage and report ABI coverage separately instead of counting only internal native helpers.
• INF-18 (P1, SEC) Privileged publish action is mutable and PR-eligible
  • 근거: changepacks/action@main at publish.yml:319; job has contents/OIDC/Pages/PR write and no push-main job condition (141-148,289-322).
  • 수정 방향: pin SHA; split validation from a push-main/environment-protected publish job with minimal permissions.
• INF-19 (P1, MAINT) CI toolchain is mutable/deprecated
  • 근거: KNOWN-19 jetli/wasm-pack-action@v0.4.0, version: latest, ubuntu-latest, and remote cargo-binstall shell installer (publish.yml:45-46,100-101,160-187).
  • 수정 방향: pin runner/tool/action versions and checksum a fixed cargo-binstall release.
• INF-21 (P2, TEST) E2E is broad for landing but narrow for release compatibility
  • 근거: Chromium-only static landing runs; no npm-packed consumer, browser matrix, or each-bundler consumer (playwright.config.ts:3-49).
  • 수정 방향: add small tarball consumer smoke projects for CJS/ESM and each plugin; add at least WebKit/Firefox where CSS/browser compatibility matters.
• INF-22 (P2, SEC) Validation job can create and commit snapshots
  • 근거: missing snapshots trigger generation/commit (publish.yml:257-271) in the same write-capable job.
  • 수정 방향: fail with uploaded artifacts or use a separate explicit maintainer workflow.
• INF-25 (P3, TEST) CI jobs can hang indefinitely
  • 근거: no timeout-minutes on any of three jobs (publish.yml:18-322).
  • 수정 방향: set realistic per-job limits above measured normal duration.

선택 (13)

• NXT-21 (P1, TEST) Benchmark CI cannot catch a performance regression
  • 근거: .github/workflows/publish.yml:55-56 runs bun run benchmark; benchmark.js:74-104,126-139 only prints times/sizes and has no baseline, threshold, assertion, or nonzero regression exit.
  • 수정 방향: Persist a baseline artifact, compare medians/output/CSS sizes with explicit tolerances, and fail only statistically/materially significant regressions.
• INF-02 (P2, TEST) Global test cleanup is manual and non-transactional
  • 근거: prefix/debug/atom-hoist/custom-shorthand and four related maps are reset independently; a panic before cleanup leaks state, and custom shorthands have no named reset API.
  • 수정 방향: introduce one RAII test-state guard/snapshot restoring all globals on drop.
• INF-05 (P2, TEST) Bun's “100%” is a scoped metric that omits production behavior
  • 근거: bunfig.toml:2-11 excludes pkg, dist, and bun-plugin/src/plugin.ts; the regression harness bypasses root coverage.
  • 수정 방향: document scoped coverage and add artifact/plugin registration integration gates.
• INF-09 (P2, MAINT) Removed lite engine remains as stale generated artifacts
  • 근거: removal changepack dated 2026-09-29 says export/API removed, but pkg/lite/index.d.ts still exports hasBuildTimeValues; stale WASM is 1,501,575 bytes.
  • 수정 방향: remove stale directory in the next legitimate generated-package build and add a release check that forbidden removed subpaths are absent.
• INF-16 (P2, PERF) Benchmark CI cannot detect regressions
  • 근거: benchmark.js prints times/sizes and medians but compares against no baseline or threshold; job passes if commands run.
  • 수정 방향: store machine-normalized baselines and gate statistically significant regressions with an explicit tolerance.
• INF-17 (P2, PERF) SemanticBuilder pass cost is still not isolated
  • 근거: conditional pass at visit.rs:1062-1069 and error-only pass at 418-448; no dedicated criterion group. Fresh-process style-value fixture measured 4.058 ms median vs 2.732 ms control (+1.326 ms, +48.5%), but the fixture also has 51 more bytes and performs extraction, so this is only an upper bound.
  • 수정 방향: criterion benchmarks over the same parsed program with pass on/off instrumentation, separately for style_values and report_compiled_reads.
• INF-20 (P2, TEST) CI cache keys omit relevant toolchain dimensions
  • 근거: keys include OS/lockfile but omit runner image and Rust/Bun/Node/wasm-pack/Playwright versions (publish.yml:26-44,81-122,165-184,249-256).
  • 수정 방향: include explicit runtime/tool versions and browser revision in keys.
• INF-26 (P2, PERF) WASM size has no tracked trend or budget
  • 근거: current full WASM is 5,387,595 bytes and ignored generated binary has no git log history; CI verifies loadability but not size.
  • 수정 방향: record compressed/uncompressed artifact size in CI, compare to main, and fail above an approved budget.
• NXT-22 (P2, TEST) Benchmark sampling policy makes most rows incomparable and flaky
  • 근거: fixture list is at benchmark.js:86-104, but only three turboTargets get six alternating samples/medians at benchmark.js:110-123; Webpack and other fixtures are one cold observation.
  • 수정 방향: Define targets and sampling policy in a manifest; warm/alternate consistently and report spread plus median for every published comparison.
• NXT-23 (P2, MISMATCH) English and Korean benchmark tables are from different runs
  • 근거: README.md:82-104 links run 33254030962; README_ko.md:82-104 links run 33239265133 and reports different values for the same rows.
  • 수정 방향: Generate both tables and links from one checked benchmark result artifact.
• PLG-26 (P2, TEST) Important plugin behavior is untested despite 100% line coverage
  • 근거: Bun tests simulate private helper logic and assert spies are “defined”; they never invoke captured onLoad callbacks, so PLG-01/02/03-extension behavior passes. Rsbuild tests mock both mkdir/write as immediately resolved and miss PLG-10. No suite covers deletion/rename, extends watches, same-length CSS edits, multi-environment order, or write reordering.
  • 수정 방향: Add hook-level mocked-context tests plus small end-to-end bundler fixtures for CSS assets, HMR removal, cold directories, and concurrent revisions.
• INF-24 (P3, MAINT) WASM package documentation is stale template content
  • 근거: binding and packaged READMEs still say wasm-pack-template and show obsolete Travis/template setup.
  • 수정 방향: replace generated-package README input with current API/usage/error/state notes.
• NXT-24 (P3, MISMATCH) README “Build Size” hides the separately measured CSS size
  • 근거: benchmark.js:68-70 computes total output and CSS output separately, but README.md:86 and README_ko.md:86 publish only an ambiguous “Build Size” total.
  • 수정 방향: Publish separate total-output and CSS-byte columns with the measured directories/files.