Hatch rollback and remove refuse with "configuration drifted" after the project version is bumped or a dependency is added, in both hosted and vendored mode

[agent] Found by the scheduled Hatch bug-hunt routine (ledger #314).

Summary

For a Hatch project, socket-patch records whole pyproject.toml snapshots (hatch_document wiring and the HatchDocument hosted ledger edit) and reverts them with the three-way TOML merge vendor::pypi_lock::restore_document. That merge was written for lock [[package]] tables, and it treats two ordinary pyproject edits as drift:

[project].version (or name) changed. restore_item / restore_value call same_identity, which requires the live table's name and version to equal the recorded ones. Applied to the [project] table, that fails as soon as the project releases a new version. Even a trailing comment on name = "app" trips it, because the comparison is textual.
Any element added to or removed from the array that holds the patched pin (for example a new entry in project.dependencies). restore_value returns drift when live.len() != new.len().

After either edit, socket-patch rollback and socket-patch remove refuse, and the patch stays wired. A re-scan (scan --mode hosted or --mode vendored) exits 0 but doesn't re-baseline the ledger, so the next rollback fails the same way. Edits in other tables, such as a new env dependency or a new [tool.ruff] table, merge fine.

Impact

Bumping [project].version is part of every release, and adding a dependency is routine. After either, a Hatch project can't unpatch through socket-patch in either mode. remove exits with hosted_revert_failed / vendor_revert_failed, and the only way out is to edit pyproject.toml by hand. That includes removing allow-direct-references correctly, which socket-patch otherwise tracks ownership of. It fails closed (nothing is corrupted), but it blocks the documented rollback and remove workflow.

Repro (Linux, Hatch 1.18.1 and 1.7.0, main `f6b7fb9`)

Mock patch API serving a patched six 1.16.0 wheel: the same mock as the #335 probe (https://github.com/SocketDev/socket-patch/actions/runs/36740025279), modeled on tests/vex_pypi_real_common.

SP="socket-patch --api-url http://127.0.0.1:18080 --api-token fake --org test-org --patch-server-url http://127.0.0.1:18080"
mkdir -p app/src/app && cd app && touch src/app/__init__.py
cat > pyproject.toml <<'EOF'
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "app"
version = "0.1.0"
dependencies = ["six==1.16.0"]

[tool.hatch.build.targets.wheel]
packages = ["src/app"]
EOF
hatch env create
VIRTUAL_ENV=$(hatch env find default) $SP scan --mode hosted --json --yes --ecosystems pypi   # redirected: 1
sed -i 's/0.1.0/0.2.0/' pyproject.toml                  # a release bump
$SP rollback --json      # exit 1, partial_failure: hosted.failed[0].error = "pyproject.toml: Hatch configuration drifted (pyproject.toml)"
$SP remove pkg:pypi/six@1.16.0 --json --yes   # hosted_revert_failed, same message
$SP scan --mode hosted --json --yes --ecosystems pypi && $SP rollback --json   # still exit 1
grep -c 'six @' pyproject.toml                          # 1: still wired

With --mode vendored --vendor-source build (Hatch on PATH), the same steps give vendoredFailed: "pyproject.toml changed since patching", and remove gives vendor_revert_failed.

Control: without the edit, rollback restores pyproject.toml byte for byte in both modes (this passed on the previous run for 13 hosted and 12 vendored shapes).

Expected vs actual

Expected: docs/testing/hatch.md says both modes "record reversible document edits", and that selective and preserved rollback restore allow-direct-references after the last reference is unwired. The only refusals it documents are drifted sources, ledgerless direct references, concurrent edits and symlinks. An unrelated key in [project], or a sibling entry in the dependency array, is none of those, so rollback should put six==1.16.0 back, keep the user's edits, and drop the permission.
Actual: rollback and remove refuse with a drift error on every run, and the re-scan doesn't recover.

Matrix (Linux)

Edit after patching	hosted 1.18.1	vendored 1.18.1	hosted 1.7.0	vendored 1.7.0
`[project].version` bump	fail	fail	fail	fail
add `"idna==3.7"` to `project.dependencies`	fail	fail	untested	untested
comment appended to `name = "app"`	fail	untested	untested	untested
add an env dependency in `[tool.hatch.envs.default]`	pass	untested	untested	untested
append a `[tool.ruff]` table	pass	untested	untested	untested
version bump, then re-scan, then rollback	fail	untested	fail	fail

The failure comes from socket-patch's TOML merge, not from Hatch, so it's the same on every Hatch version and OS. No macOS or Windows probe was run for that reason.

First bad

Hatch support landed in #244 (649d457), which is after v4.0.0, so no release is affected yet. It has been present since Hatch support landed.

Suspect code

crates/socket-patch-core/src/vendor/pypi_lock.rs:400 same_identity (called at :462 and :495) applies the lock-package identity check to pyproject tables.
crates/socket-patch-core/src/vendor/pypi_lock.rs:470 / :505 treat any array-length change as drift.
The callers are crates/socket-patch-core/src/patch/redirect/replay.rs:652 (hosted HatchDocument) and crates/socket-patch-core/src/vendor/pypi_hatch.rs:253 (vendored revert).

Related, but a different code path: #379 (uv, ReplaceFragment whole-document fragments) and #382 (PDM re-scan never normalizes the ledger).

[agent] Triaged: priority:p1 (Hatch / PyPI family). Not a duplicate, and no open or merged PR fixes it.

Cause confirmed on main: the Hatch pyproject.toml revert (vendored vendor/pypi_hatch.rs and hosted patch/redirect/replay.rs / utils/hatch.rs) goes through vendor::pypi_lock::restore_document. That three-way merge was written for lock [[package]] tables, so it calls same_identity (textual name + version equality) on every table-like it descends into, including [project], and it treats any array length change as drift. It's related to #379 and #382 (hosted rollback stuck after an unrelated edit), but those fail on the uv/PDM ledger fragment and re-scan rebase, which is a different code path, so they aren't clustered.

Generated by Claude Code

[agent] Re-triage on main 2463257 (#277, the v5 workflow), Linux, Hatch 1.18.1, same mock patch API. Half fixed:

Edit after patching	hosted	vendored
`[project].version` bump	fixed: rollback `success`, `six==1.16.0` restored, version edit kept, `[tool.hatch.metadata]` dropped	still fails
add `"idna==3.7"` to `project.dependencies`	fixed: restored `["idna==3.7", "six==1.16.0"]`	still fails
comment appended to `name = "app"`	fixed	untested

In v5 hosted rollback restores the upstream pin from the live file, so it no longer goes through the snapshot merge. The vendored path still does:

socket-patch scan --mode vendored --json --yes …   # pyproject wired to {root:uri}/.socket/vendor/pypi/<uuid>/six-1.16.0-…whl
sed -i 's/0.1.0/0.3.0/' pyproject.toml              # or add a dependency to the same array
socket-patch rollback --json …
#  -> exit 1, partial_failure, vendoredFailed: [{"purl":"pkg:pypi/six@1.16.0","error":"pyproject.toml changed since patching"}]
socket-patch remove pkg:pypi/six@1.16.0 --json --yes …
#  -> error vendor_revert_failed "could not revert vendoring for pkg:pypi/six@1.16.0: pyproject.toml changed since patching"

Reproduced 3 times (version bump ×2, added dependency ×1). Suspect code on current main: crates/socket-patch-core/src/vendor/pypi_hatch.rs:258 → vendor/pypi_lock.rs:534 restore_document, with same_identity at :408 and the array-length checks at :478 / :513.

Generated by Claude Code

[agent] Shares root cause with #474: the vendored revert's three-way TOML merge (restore_document in crates/socket-patch-core/src/vendor/pypi_lock.rs) matches array elements by position and treats any array-length change as drift, and applies the lock-package name/version identity check to every table (including pyproject [project]). Will be fixed together.

Generated by Claude Code

[agent] Claiming this issue (with #474; shared root cause: positional/length-strict array merge and table-level identity check in vendor::pypi_lock::restore_document). Branch: agent/fix-pypi-toml-merge-identity. Claim-ID: 2026-10-01T16:20:43Z-576d58

Generated by Claude Code

[agent] Draft PR: #481

Generated by Claude Code

Let vendored Python reverts keep sibling edits

Test real uv/Hatch reverts after sibling edits

[agent] Re-triage on main 61cfb9b (Linux, real Hatch 1.18.1, mock patch API):

Hosted: passes now. After a [project].version bump, and after adding a dependency to the array that holds the pin, rollback --json gives success with hosted.reverted: [pkg:pypi/six@1.16.0]. six==1.16.0 comes back and the unrelated edit is kept.
Vendored: still fails. After the version bump, rollback --json gives partial_failure with vendoredFailed: [{purl: pkg:pypi/six@1.16.0, error: "pyproject.toml changed since patching"}], and the {root:uri} wiring stays. Reproduced 2×.

So what's left of this issue is the vendored ledger (vendor/pypi_hatch.rs revert / ledger_snapshots).

Generated by Claude Code

Fix vendored Python reverts refusing sibling edits (#385, #474) (#481)

Navigation Menu

Repository navigation

Hatch rollback and remove refuse with "configuration drifted" after the project version is bumped or a dependency is added, in both hosted and vendored mode #385

Summary

Impact

Repro (Linux, Hatch 1.18.1 and 1.7.0, main `f6b7fb9`)

Expected vs actual

Matrix (Linux)

First bad

Suspect code

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Hatch rollback and remove refuse with "configuration drifted" after the project version is bumped or a dependency is added, in both hosted and vendored mode #385

Description

Summary

Impact

Repro (Linux, Hatch 1.18.1 and 1.7.0, main f6b7fb9)

Expected vs actual

Matrix (Linux)

First bad

Suspect code

Activity

mikolalysenko commented on Sep 30, 2026

mikolalysenko commented on Oct 1, 2026

mikolalysenko commented on Oct 1, 2026

mikolalysenko commented on Oct 1, 2026

mikolalysenko commented on Oct 1, 2026

mikolalysenko commented on Oct 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Repro (Linux, Hatch 1.18.1 and 1.7.0, main `f6b7fb9`)