Repository navigation
Preserve unrelated errors during pending asset run creation - #75523
Open
ataulmujeeb-cyber wants to merge 1 commit into
Open
ataulmujeeb-cyber wants to merge 1 commit into
ataulmujeeb-cyber wants to merge 1 commit into
Conversation
This was referenced Oct 23, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Related: #71074
AssetManager._get_or_create_apdrtreats everyIntegrityErrorraised while creating a pendingAssetPartitionDagRunas "another producer inserted this row first". Only one constraint actually means that, so unrelated database errors get pulled into the race-recovery path.Root Cause
The recovery branch catches the base exception class:
Any constraint failure reaches it. The damaging case is when the error comes from the caller's own pending rows rather than from the new one, because
Session.begin_nested()flushes pending ORM state before opening its savepoint:NOT NULL, then calls into asset-change registration.begin_nested()flushes it, the database rejects it, and the transaction is now in a failed state.PendingRollbackError.The original error is replaced by one that points somewhere else, and an error that retrying cannot fix is retried first.
NOT NULL/ PK / otherUNIQUEFix
Classify the error from driver diagnostics instead of the rendered message, and recover only for the pending-marker constraint:
_is_pending_marker_conflictreads SQLSTATE23505plus the constraint name on PostgreSQL,sqlite_errorcode2067 with the column list on SQLite, and errno 1062 with the trailingfor keyname on MySQL.Diagnostics rather than text matters because the rendered
IntegrityErrorincludes the statement and its bound parameters. An ordinary row whose value happens to contain the constraint name, or a MySQL value carrying its ownfor key '...'fragment, is enough to make an unrelated failure look like the marker conflict.Genuine marker conflicts keep the existing two-attempt recovery and the existing exhaustion behaviour of re-raising the driver's
IntegrityError. No migration, no schema change.Changes
airflow-core/src/airflow/assets/manager.py: added_is_pending_marker_conflictand narrowed the recovery branch in_get_or_create_apdrto that constraint.airflow-core/tests/unit/assets/test_marker_conflict_classifier.py: new module, 11 tests driven by real database errors:NOT NULLand unrelatedUNIQUEviolations all propagatefor key '...'fragment_get_or_create_apdron the first attemptPendingRollbackErrorTesting
Earlier runs of the same change on an equivalent tree: 22 passed on psycopg3 for the wider focused selection; 11 passed on psycopg2, mysqlclient and PyMySQL each; 21 passed on SQLite with one pre-existing concurrency test skipped.
Restoring the bare
except IntegrityErrormakes the two negative helper tests fail on PostgreSQL/psycopg3, MySQL/mysqlclient and SQLite: the caller-flush case performs three lookups instead of one and surfacesPendingRollbackErrorin place of the original error.The tests provoke real constraint violations rather than constructing exceptions.
test_helper_recovers_from_a_real_conflict_end_to_endforces the first lookup to miss so the insert hits a genuine unique violation; it is a staged single-transaction control, not a two-transaction race. Obscuring the original error is a diagnostic problem and these tests do not demonstrate data loss. SQLite results do not establish concurrent-writer behaviour.Affected Components
AssetManager._get_or_create_apdrbatch_asset_eventsRelated Follow-ups
Each stands alone and can be reviewed independently.