Skip to content

Bench: cover Gradle and Hatch hosted modes - #925

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
bench/refresh
Oct 7, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
bench/refresh

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

What main changed. Two hosted rewriters landed without bench coverage.

Gradle: #646 (0685ba8c) added full Gradle support, including hosted mode. scan now crawls Gradle's modules-2/files-2.1 cache under GRADLE_USER_HOME and pins the suffixed versions in gradle.lockfile. It wires the build through an owned settings script and index under .socket/gradle/. The suite had no Gradle scenario: maven/* only covers the pom.xml + ~/.m2 path.

Hatch: #680/#743 (0475695f) added hosted Hatch. With a Hatch project (hatch.toml, [tool.hatch] or a hatchling.build backend) and no lockfile, scan rewrites each declared dep in place through utils::hatch::plan: it turns pyproject.toml [project] deps and hatch.toml [envs.*] deps into name @ <url>#sha256=… and enables allow-direct-references. hatch.toml is a HOSTED pypi input in formats/registry.rs, but no fixture reached this path.

Suite changes

  • Added hatch/hosted and hatch/rescan (commit 5b9ae8ea, 2026-10-07). The project is a lockless hatchling app with 1000 dists and 25 patched. Direct deps go in [project]. A hatch.toml default env (in-project .venv) pins every patched dist, plus every 10th other one, so the planner has to skip non-matching specs. Hosted Hatch only redirects deps a Hatch table declares (a transitive-only dep gets redirect_hatch_unsupported), so these pins are what a real project patching its transitives writes. The scenarios expect hatch.toml and pyproject.toml to be rewritten, and the only allowed warning is the shared redirect_pypi_stale_install.
  • Added gradle/hosted and gradle/rescan. The project is a single-project Groovy-DSL build with dependencyLocking: 1000 locked artifacts, 25 of them patched direct deps. Its cache lives under the fixture's GRADLE_USER_HOME, with jar and pom in separate sha1 hash dirs. The scenarios expect these rewrites: .socket/gradle/{.gitattributes,hosted-index.tsv,socket-patch.hosted.settings.gradle}, gradle.lockfile and settings.gradle. The only allowed warning is redirect_gradle_detached_configs_unguarded, which the planner always emits.
  • The grant's indexUrl is https://patch.socket.dev/..., not the mock. The Gradle planner refuses non-https repositories (redirect_gradle_override_invalid), and a scan never fetches the index. The artifact URL still points at the mock.
  • Refactor, no behavior change. maven and gradle now share the Maven-coordinate generator, the pom writer and the maven2 grant builder. The maven fixture bytes are unchanged: I built maven/hosted with the old and new bench binaries and diff -r showed them identical.
  • README: lists gradle and GRADLE_USER_HOME.

Validation, Gradle (all runs against the main CLI 9c43dfc9, on a 4 vCPU Xeon @ 2.10GHz)

  • cargo fmt -p socket-patch-bench, cargo clippy -p socket-patch-bench --all-features --all-targets -- -D warnings and cargo test -p socket-patch-bench (29 passed) are all clean.
  • run -f '^gradle/' --runs 3: gradle/hosted 170.5 ms (0.17 ms/pkg), gradle/rescan 259.2 ms. Both validate with 53 requests each.
  • A/A compare -f '^gradle/' -f '^maven/' (head binary vs a copy of itself): no regressions. gradle/hosted -1.1%, gradle/rescan +4.5%, maven/hosted -1.6%, maven/rescan +1.7%.
  • strace -f -e trace=execve on the serve gradle/hosted command: the CLI spawns nothing; the only execves are the shell, env and socket-patch itself. The rewritten gradle.lockfile carries <v>-socket.<uuid8> pins, and settings.gradle gains the apply from: line.

Validation, Hatch (main CLI 9c43dfc9, 4 vCPU Xeon @ 2.80GHz, 2026-10-07)

  • cargo fmt -p socket-patch-bench, cargo clippy -p socket-patch-bench --all-features --all-targets -- -D warnings and cargo test -p socket-patch-bench (29 passed) are all clean.
  • run -f '^hatch/' --runs 5: hatch/hosted 86.1 ms (0.086 ms/pkg), hatch/rescan 77.0 ms. Both validate with 53 requests and 25 redirected. At 400/12 a scan took about 50 ms, under the suite's ~70 ms floor, so the size was raised to 1000/25.
  • A/A compare -f '^hatch/': no regressions. hatch/hosted -2.1%, hatch/rescan +2.5%.
  • strace -f -e trace=execve on the serve hatch/hosted command: the only execve is socket-patch itself. The rewritten hatch.toml holds name @ http://…/…whl#sha256=… pins, and pyproject.toml gains [tool.hatch.metadata] allow-direct-references = true.

Time budget: compare grows by about 2 × 0.4 s per pair. A full compare took about 13 min on this runner both before and after, so the change is within noise. Hatch adds about 2 × 0.16 s per pair, roughly 6 s per full compare. Nothing was removed.

🤖 Generated with Claude Code

https://claude.ai/code/session_019rfKYrfN4H9XDnDS8MYaYJ


Note

Low Risk
Changes are benchmark fixtures, docs, and digest helper consolidation with no intended production behavior change.

Overview
Adds socket-patch-bench coverage for hosted Hatch and Gradle scan paths, which previously had no benchmark scenarios.

Hatch gets a lockless hatchling fixture (hatch.toml env pins + pyproject.toml) with hosted/rescan expectations on rewriting both files. Gradle gets a locked gradle.lockfile build with a synthetic GRADLE_USER_HOME cache, expecting .socket/gradle/ hosted wiring plus settings.gradle and lockfile updates; patch grants use an https:// registry index URL because Gradle rejects non-HTTPS repos.

Maven fixture generation is refactored to share JVM coordinate universe, POM, and maven2 patch builders with Gradle; Maven fixture bytes are intended to stay identical. The bench README documents hatch, gradle, and GRADLE_USER_HOME isolation.

In socket-patch-core, Gradle cache, JVM jar patching, and Maven sidecar checksum logic now call shared utils::digest helpers instead of inlining sha1/sha2 hashing (refactor only).

Reviewed by Cursor Bugbot for commit 5b9ae8e. Configure here.


Generated by Claude Code

#646 gave Gradle builds a hosted mode: scan crawls Gradle's
modules-2/files-2.1 cache, pins suffixed versions in gradle.lockfile
and wires the build through an owned settings script and index under
.socket/gradle/. None of that was benchmarked; the maven scenarios
only reach the pom.xml + ~/.m2 path.

The gradle fixture is a single-project Groovy build with dependency
locking (1000 locked artifacts, 25 patched direct deps), its cache
under the fixture's GRADLE_USER_HOME with jar and pom in separate
sha1 dirs. The Maven-coordinate generator, pom writer and maven2
grant builder are shared with the maven fixture, whose bytes are
unchanged. The grant's indexUrl is https because the Gradle planner
refuses anything else; scan never fetches it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mikolalysenko Mikola Lysenko (mikolalysenko) added the bench socket-patch scan benchmark suite label Oct 6, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

main has failed socket-patch-core's lib tests since Gradle support
(#646) and the digest helpers (#865) both landed. The guard test
production_digests_go_through_the_helpers flags three files #646 added
that still hash inline: crawlers/gradle_cache.rs, patch/jvm_jar.rs and
patch/sidecars/maven.rs. That breaks test, test-release and coverage on
every open PR.

Each inline sha1/sha256 call now goes through sha1_hex_of or
sha256_hex_of, which compute the same lowercase hex. Behaviour is
unchanged.

Assisted-by: Claude Code:claude-opus-5-5
(cherry picked from commit 659ac2c)
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] Bench: coverage failed on 9b5c5491 because of main, not this PR.

utils::digest::tests::production_digests_go_through_the_helpers fails because #865 (digest helpers) and #646 (Gradle) crossed. Gradle added inline digests in crawlers/gradle_cache.rs, patch/jvm_jar.rs and patch/sidecars/maven.rs. The same test is red in main's CI on 9c43dfc9.

The fix is open as #878. I cherry-picked its single commit here (659ac2c2, with -x) so this PR can go green. It becomes a no-op once #878 merges. Locally, cargo test -p socket-patch-core --lib digest (21 passed) and cargo clippy -p socket-patch-core --lib -D warnings are both clean.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 6, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Burn-down agent: labeled Ready for review.


Generated by Claude Code

Hatch hosted mode (#680, #743) rewrites pyproject.toml and hatch.toml
in place, with no lockfile, through utils::hatch::plan. No scenario
exercised that rewriter: hatch.toml is a HOSTED pypi input, and a
hatch project with no lock fell through every existing pypi fixture.

The fixture is a lockless hatchling app. Direct deps go in [project],
and a hatch.toml default env (in-project .venv) pins every patched
transitive, since hosted Hatch only redirects deps a Hatch table
declares. A scan rewrites both files and adds
[tool.hatch.metadata] allow-direct-references. It is sized at 1000
packages / 25 patched so a scan takes about 75-85 ms.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mikolalysenko Mikola Lysenko (mikolalysenko) changed the title Bench: cover Gradle hosted mode (gradle/hosted, gradle/rescan) Bench: cover Gradle and Hatch hosted modes Oct 7, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 5b9ae8e. Configure here.

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] Ready for review at 5b9ae8ea (re-verified after the Hatch bench commit landed on top of the previously labeled bb74cac).

  • CI: all check runs green (success/skipped) on 5b9ae8ea; mergeable, clean.
  • Bugbot: reviewed 5b9ae8ea, no unresolved findings.
  • Reviewer note: the new hatch/hosted and hatch/rescan scenarios are the only change since the last label.

Generated by Claude Code

@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit 7c46ac3 into main Oct 7, 2026
412 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the bench/refresh branch October 7, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bench socket-patch scan benchmark suite Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants