Fix requirements -r after other options (#1028)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1028

Root cause

include_target_with (crates/socket-patch-core/src/vendor/pypi_requirements.rs) only matched the first shlex word of a requirements line against -r/--requirement. pip runs optparse over every option word (break_args_options → shlex.split → optparse, then follows opts.requirements[0]), so --pre -r dev.txt, -i URL -r dev.txt, -c c.txt -r dev.txt and --prefer-binary --requirement=dev.txt are includes pip follows. Lock-only discovery, the vendored planner walk (walk_requirements_tree), requirements_include_names and the lock inventory (lock_inventory/pypi.rs) all go through this one function, so all of them skipped those includes: a lock-only scan exited 0 with "No patches" while pip installed the include's unpatched pins.

Fix

Parse the option part of the line the way pip's optparse does:

• a line whose first word doesn't start with - is a requirement (its per-requirement options never recurse);
• walk every option word; consume the value of each value-taking req-file option (-i, --extra-index-url, -c, -f, --no-binary, --only-binary, --trusted-host, --hash, ...), so -i -r dev.txt is an index URL, as in pip;
• resolve optparse's unique long-option prefixes (--requirem → --requirement);
• the first -r value wins; an -e anywhere makes the line an editable requirement; -- ends options.

Existing #994 behaviour (quotes, ${VAR}, attached -rfile, --requirement= dev.txt) is unchanged. The npm/pypi/gem wrappers only dispatch to the binary; nothing to mirror there.

Tests (red → green)

Issue	Test	Without fix	With fix
#1028	core include_target_scans_every_option_word (26 cases incl. value-looks-like--r, requirement line, editable, --)	FAIL ("--pre -r dev.txt": None)	ok
#1028	core include_after_other_option_is_followed (planner wires the pin in dev.txt, root untouched)	FAIL	ok
#1028	CLI scan_requirements_lock_only::lock_only_scan_discovers_include_after_other_options (6 line shapes from the issue, hosted + --vendor)	FAIL (lockfileOnlyPackages 0 vs 1)	ok

Local runs:

• cargo clippy --workspace --all-features -- -D warnings: clean.
• rustfmt --edition 2021 --check on both changed files: clean. (cargo fmt --all -- --check reports pre-existing diffs on untouched files on main too; CI does not run it.)
• cargo test -p socket-patch-core --all-features --lib: 5560 passed, 4 failed. The 4 (relax_loop_must_not_traverse_symlinked_root, an_unremovable_hidden_lock_keeps_every_store_entry, wire_write_failure_maps_error_and_leaves_lock_untouched, wire_failure_rolls_back_already_written_files) depend on chmod making writes fail, which doesn't happen in this sandbox because it runs as root. They don't touch the changed function.
• CLI: scan_requirements_lock_only 5/5, scan_vendor_requirements_unwired 3/3, in_process_vendor_pypi_takeover 6/6, mode_migration_pypi 31/31, e2e_vendor_pypi_build -- --include-ignored 36/36 (real pip 24.0 / uv).
• A full local cargo test --workspace couldn't finish: the sandbox ran out of disk linking the CLI test binaries. CI runs the full suite.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mh4a2b5NdLiBUqR1vaAzeQ

---

Note

Medium Risk
Changes shared requirements include discovery used by scan, lock inventory, and vendored wiring; behavior is narrowed to match pip but affects all PyPI lock-only paths.

Overview
Fixes #1028: lock-only scan and requirements-tree walking no longer miss -r includes when another pip option appears first on the same line (e.g. --pre -r dev.txt, -i URL -r dev.txt).

include_target_with in pypi_requirements.rs is rewritten to mirror pip’s requirements-file optparse: skip requirement lines whose first token isn’t an option; walk every option word; consume values for known value-taking flags; resolve unique long-option prefixes; treat -e as a requirement line; honor --; take the first -r/--requirement as the include target (and don’t treat -r inside another option’s value as an include).

Adds unit tests for the parser edge cases, an async test that the planner wires pins through such includes without duplicating the root file, and a CLI lock-only scan test covering six line shapes in hosted and --vendor mode.

^{Reviewed by Cursor Bugbot for commit a2d873e. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

About 26 e2e (ubuntu-latest, …) matrix jobs on 2fa4021 failed between 16:58 and 16:59Z without ever running. They span every ecosystem: vlt, gem, composer, maven, gradle, sbt, nuget, deno, hatch/pdm vex, and also e2e_vendor_pypi_build 0.12.17.

All of them were created at 16:39Z and failed about 20 minutes later. None got a runner: the job API shows runner_id: 0 and no runner name, and they have no log (404) and no check output. That's runner starvation on this run, not test failures. The one PyPI job in the batch failed the same way. This PR's change is covered by e2e_vendor_pypi_build, which passed 36/36 locally against real pip 24.0 and uv.

The CI run is still in progress, so failed jobs can't be re-run yet. I'll re-run them once, when it finishes. Anything that then fails after actually running tests gets root-caused here.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a2d873e. Configure here.}

[Mikola Lysenko (mikolalysenko)]

[agent] Ready for review at a2d873e.

• What changed: merged current origin/main into the branch with no conflicts, so the head has the merge-queue ci-ok job. There are no code changes beyond the merge. Locally, the CI clippy command (--workspace --all-features -D warnings) on Linux-equivalent code, rustfmt on the touched files, the pypi_requirements unit tests (47 passed) and scan_requirements_lock_only (5 passed) all pass.
• CI on the head: 502 check runs: 496 passed, 6 skipped, 0 failed. ci-ok passed. The earlier runner-starvation failures on 2fa4021 did not recur. The Bun and vlt macOS/Windows matrices were queued for a long time because of a repo-wide runner backlog, but all of them eventually passed.
• Bugbot: ran on a2d873ef. It found no new issues and there are no open review threads.
• Mergeable: clean. I added the Ready for review label.

Generated by Claude Code