Fix Pipenv project falling back to system Python (#504, #947)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #504
Fixes #947

Summary

A Pipenv project with no Pipenv venv yet no longer has the OS Python's site-packages crawled as if they belonged to the project. This covers a fresh checkout before pipenv install, and a project that only has a plain venv/ or an opted-out .venv:

• Agent-mode scan in a Pipenv project without a Pipenv venv patches the system Python's site-packages in place instead of the project's venv/ (regression from #388) #504: agent mode no longer patches the system interpreter in place, fails with EACCES, or lets VEX attest not_affected for it.
• Vendored scan of a fresh Pipenv checkout (no venv yet) fails with exit 1 on packages that exist only in the system Python, because the crawler falls back to the global site-packages #947: a vendored scan of a fresh checkout no longer tries to vendor system-only packages into Pipfile.lock and exit 1. Hosted no longer warns about them, and --dry-run no longer promises them.

Root cause

PythonCrawler::get_site_packages_paths falls back to get_global_python_site_packages() whenever find_local_venv_site_packages returns nothing and is_python_project(cwd) holds. For a Pipenv project, the Pipenv branch returns only the venv Pipenv itself resolves (#388), so an empty result there is final: Pipenv has no venv yet and nothing is installed for the project. The caller couldn't tell that apart from "nothing probed", so it fell through to the global interpreters.

Fix

get_site_packages_paths now returns nothing for a Pipenv project once Pipenv's own venv lookup came back empty (crates/socket-patch-core/src/crawlers/python_crawler.rs). It is a single boundary: agent, hosted and vendored modes, rollback and the dispatch locator all crawl through it. Lock-only packages still join discovery from Pipfile.lock through the lockfile supplement, which the #947 test asserts. -g / --global-prefix are unchanged. No wrapper changes are needed under npm/, pypi/ or gem/, because they only dispatch to the binary.

Whether a plain ./venv should be patched again for a Pipfile project is still the maintainer decision raised in #504. This PR keeps the #388 rule (Pipenv never uses venv/), and the venv/ cell now patches nothing instead of the system Python.

Test changed on purpose: get_site_packages_paths_falls_back_via_pipfile_marker pinned the defect: it asserted that a Pipfile marker triggers the global fallback. It is replaced by get_site_packages_paths_pipenv_without_venv_never_falls_back_to_global, which asserts the opposite. Its original concern (a fresh Pipenv clone finding zero packages) is now covered by the lockfile supplement, not by the OS Python. The pyproject and uv.lock fallback tests are untouched.

Ported main fix: a21968e cherry-picks #878 ("Route Gradle digests through utils::digest"). main currently fails utils::digest::tests::production_digests_go_through_the_helpers, and that turned test (macos-latest) and coverage red on this PR. The commit no-ops once #878 lands.

Per-issue checklist

• Agent-mode scan in a Pipenv project without a Pipenv venv patches the system Python's site-packages in place instead of the project's venv/ (regression from #388) #504 → in_process_python_envs::pipenv_without_a_venv_never_scans_the_system_python, agent mode, with and without a stray venv/, plus the core test's three marker shapes.
• Vendored scan of a fresh Pipenv checkout (no venv yet) fails with exit 1 on packages that exist only in the system Python, because the crawler falls back to the global site-packages #947 → in_process_python_envs::pipenv_fresh_checkout_candidates_come_from_the_lock_only, vendored and hosted. The lock's urllib3 is discovered and the system-only package isn't.

Test evidence

• Red before the fix (main 9c43dfc): the core test got ["/usr/local/lib/python3.11/dist-packages", "/usr/lib/python3/dist-packages", …, "<HOME>/anaconda3/lib/python3.11/site-packages"]. Both CLI tests failed with unexpectedly discovered pkg:pypi/system-decoy@6.6.6, and the batch query also carried the whole system Python (pyyaml, cryptography, …).
• Green after the fix: cargo test -p socket-patch-core --test crawler_python_e2e 61/61, cargo test -p socket-patch-cli --test in_process_python_envs 21/21.
• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo test --workspace --all-features --no-fail-fast: 10,820 passed, 12 failed. The 12 failures are chmod- or unremovable-file tests that can't fail when run as root (the sandbox runs as uid 0). The covgap_commands_vendor *_state_write_failure_*, in_process_redirect write-failure and vlt-heal, repair unremovable-lock and cleanup-failure tests, plus four core lib tests, were all re-run as uid 65534 and pass.
• Pipenv e2e, as CI runs it: cargo test -p socket-patch-cli --all-features --test e2e_vex_build -- pipenv:: --ignored with SOCKET_PATCH_PIPENV_E2E_REQUIRED=1, for 2026.8.0 (Python 3.12) and 2022.12.19 (Python 3.8). Both pass. This suite runs hosted scan on a lock-only checkout, which is the Vendored scan of a fresh Pipenv checkout (no venv yet) fails with exit 1 on packages that exist only in the system Python, because the crawler falls back to the global site-packages #947 shape.
• cargo fmt --all -- --check: the hunks this PR touches are formatted. main already has 466 rustfmt diffs under the pinned 1.93.1 toolchain, and CI doesn't run fmt, so I left the rest of the tree alone.
• CI on a21968e: all 415 check runs pass (409 success, 6 skipped). Bugbot reviewed a21968e and found no issues.

🤖 Generated with Claude Code

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] test (macos-latest) and coverage failed on e7836a4 in utils::digest::tests::production_digests_go_through_the_helpers. That failure comes from main, not from this PR: #646's Gradle files hash inline, and #865's guard rejects that. I ported #878's fix (a21968e, "Route Gradle digests through utils::digest") into this branch so CI runs against a green base. The commit no-ops once #878 lands.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a21968e. Configure here.}

[Mikola Lysenko (mikolalysenko)]

[agent] Ready for review.

• Head: a21968e
• CI: all check runs green on a21968e (0 failing, 6 skipped), mergeable, 0 behind main.
• Bugbot: reviewed a21968e, no findings.
• For reviewers: commit a21968e ports Route Gradle digests through utils::digest #878's Gradle digest fix so CI runs against a green base. It no-ops once Route Gradle digests through utils::digest #878 lands. The PR keeps the Fix Pipenv venv discovery order (#334, #384) #388 rule (Pipenv never uses venv/). Whether a plain ./venv should be patched for a Pipfile project is still the open maintainer question in Agent-mode scan in a Pipenv project without a Pipenv venv patches the system Python's site-packages in place instead of the project's venv/ (regression from #388) #504.

---

Generated by Claude Code