Skip to content

Bench: benchmark NuGet as a restored project (#1183 scoped crawl) - #1250

Merged
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
bench/refresh
Oct 9, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
bench/refresh

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

[agent] Bench: daily suite refresh for main f3c6313a.

What main changed

#1183 (6f43b99c) scopes the project-mode NuGet crawl: when the project has a parsed obj/project.assets.json, the shared ~/.nuget/packages root is looked up for the restore's resolved libraries instead of walked. Every restored .NET project has that file. The nuget/* fixture did not, so the suite only timed the fallback walk.

Suite changes

  • Updated nuget/hosted, nuget/rescan: the fixture now writes obj/project.assets.json with all 900 packages as type: package libraries, so the scan takes Scope project-mode NuGet crawls to the restore's resolved packages (#427) #1183's restore-scoped lookup. packageFolders is left out (the fixture's absolute path isn't known at generation time; the crawl still finds ~/.nuget/packages through HOME). Expectations are unchanged: 900 scanned, 25 redirected, the same rewritten files.
  • README: the nuget entry describes the restored-project layout.
  • Nothing added or removed otherwise.

Validation (4 vCPU Xeon @ 2.80GHz)

  • cargo fmt -p socket-patch-bench, cargo clippy -p socket-patch-bench --all-features --all-targets -- -D warnings and cargo test -p socket-patch-bench (29 passed) are clean.
  • run -f '^nuget/' --runs 5 on main f3c6313a: both scenarios validate. nuget/hosted 118.1 ms (peak RSS 28.3 MiB), nuget/rescan 93.1 ms, both above the ~70 ms floor.
  • A/A compare -f '^nuget/' (same binary at two paths): no regressions (+6.3% [-1.1, +22.3] and -3.0%, both ≈).
  • A/B ea097142 (before Scope project-mode NuGet crawls to the restore's resolved packages (#427) #1183) → f3c6313a on the new fixture: nuget/hosted ≈, nuget/rescan -11.4% [-19.1, -1.5].
  • strace -f -e trace=execve on the serve nuget/hosted command: the CLI spawns nothing.

Time budget

No scenarios were added. Fixture build time is unchanged within noise. A full compare took 21.7 min on today's runner (45 scenarios, including 4 confirmation rounds). It is still over the ~12-minute target, as noted in #575.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XA89noGPP3ZYzoApZozfst


Generated by Claude Code


Note

Low Risk
Benchmark fixture and documentation only; no production crawl or CLI behavior changes.

Overview
Updates the NuGet benchmark fixture so scans exercise the restore-scoped crawl from #1183 instead of only the global ~/.nuget/packages walk.

build_nuget now writes project/obj/project.assets.json listing every synthetic package as a type: package library (with paths/sha512 aligned to the existing lockfile and cache layout). packageFolders is omitted because the generator does not know the fixture’s absolute path; the crawl still resolves ~/.nuget/packages via HOME. Scan expectations (900 scanned, 25 redirected, same rewritten files) are unchanged.

The README nuget bullet now describes this restored-project layout (packages.lock.json + obj/project.assets.json).

Reviewed by Cursor Bugbot for commit 37a0909. Configure here.


Generated by Claude Code

The NuGet fixture had packages.lock.json but no obj/project.assets.json,
which every restored project has. Since #1183 a project-mode crawl with
a parsed restore looks its resolved libraries up in ~/.nuget/packages
instead of walking the whole cache, so the suite only timed the
fallback walk that real restored projects no longer take.

Write a project.assets.json listing every package as a library, so
nuget/hosted and nuget/rescan exercise the restore-scoped lookup.
Expectations are unchanged: the same 900 packages are found.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mikolalysenko Mikola Lysenko (mikolalysenko) added the bench socket-patch scan benchmark suite label Oct 9, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 37a0909. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 9, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review (burn-down agent).

  • Head: 37a0909
  • CI: 222/222 green (213 success, 9 skipped)
  • Bugbot: reviewed 37a0909, no findings
  • Mergeable, no CHANGELOG.md changes.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does: The NuGet bench fixture now writes project/obj/project.assets.json listing all 900 packages, so the crawler takes #1183's restore-scoped lookup instead of walking the whole global folder. The README is updated to match; scenarios and expected results are unchanged.

Risk: low. Bench fixture and README only; build_nuget has one caller and there is no fixture cache to invalidate.

Look here:

Verified: traced the bench run: cwd project/ holds the csproj, restore_scope finds the assets file (skips obj/ in the walk), HOME is the bench's work/home, and locate_libraries' lowercased <id>/<version> matches the fixture dirs, so "900 scanned" still holds. Nothing else reads project.assets.json. CI 223/223 green on 37a0909 (ci-ok, clippy). Bugbot: no findings on this head. No CHANGELOG.md change, no open threads.

Changes I made: none.

Open questions (non-blocking): the suite no longer times the unscoped ~/.nuget/packages walk (still used in global mode and for unrestored projects). Keep a variant without the assets file if that path's perf matters.

Auto-merge is armed: approving sends this straight to the merge queue.


Generated by Claude Code

@mikolalysenko
Mikola Lysenko (mikolalysenko) added this pull request to the merge queue Oct 9, 2026
Merged via the queue into main with commit 109dd0c Oct 9, 2026
223 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the bench/refresh branch October 9, 2026 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bench socket-patch scan benchmark suite Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants