Skip to content

Allow downgrading past the TI ownership migration when data is present. - #74407

Merged
ashb merged 1 commit into
apache:mainfrom
astronomer:worktree-migration0142-downgrade
Oct 15, 2026
Merged

ashb merged 1 commit into
apache:mainfrom
astronomer:worktree-migration0142-downgrade

Conversation

@ashb

@ashb ashb commented Oct 7, 2026

Copy link
Copy Markdown
Member

The downgrade refused whenever XCom_v2 or rendered field data had rows, or
any task retried since the upgrade, so a rollback after any real use of 3.4
was impossible.

This fixes it, as long as the data can be downgraded non-destructively
"as if 3.3 wrote it". This means you might loose data that 3.3 wouldn't
have kept for archived tries.

Document the cost and caveats in the 3.4.0 release notes.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@ashb
ashb requested a review from ephraimbuddy as a code owner October 7, 2026 14:18
@boring-cyborg boring-cyborg Bot added the area:db-migrations PRs with DB migration label Oct 7, 2026
@ashb
ashb requested a review from vatsrahul1001 October 7, 2026 14:18
Comment thread RELEASE_NOTES.rst Resolved
Comment thread RELEASE_NOTES.rst Resolved
@ashb
ashb force-pushed the worktree-migration0142-downgrade branch from e71c0b3 to ad93457 Compare October 8, 2026 14:42
@ashb
ashb requested a review from XD-DENG as a code owner October 8, 2026 14:42
@ashb
ashb requested a review from kaxil October 8, 2026 14:48
@ashb ashb added this to the Airflow 3.4.0 milestone Oct 13, 2026
Comment thread RELEASE_NOTES.rst Resolved
@vatsrahul1001 vatsrahul1001 added the backport-to-v3-4-test Backport to v3-4-test label Oct 14, 2026
The 0142 downgrade refused whenever xcom_v2 or rtif_v2 held rows or any
task had retried since the upgrade, so a rollback after any real use of
3.4 was impossible. It now folds that data back into the legacy tables as
if 3.3 had written it. That is only lossless for the latest attempt: the
legacy tables are keyed by task coordinates and can hold values for one
attempt, so earlier attempts' XCom and rendered fields are dropped while
the attempts themselves move back to task_instance_history. The release
note records that cost and the runtime, which grows with the size of xcom
and rendered_task_instance_fields.

Data 3.3 cannot represent (several attempts sharing coordinates, archived
attempts with no current attempt, owners that moved while legacy rows
exist) is refused by read-only checks before the first write, because
MySQL cannot roll back DDL. Past that point a failure on MySQL leaves the
schema partly downgraded and needs a restore from backup.

The downgrade leaves try_number alone. Migration 0138's downgrade already
subtracts one from pending retries and runs after this one on any
downgrade to 3.3, so rewinding here as well made a retried task reuse an
archived try number. Only a test that downgrades through 0138 can see it.

Expansion now deletes the archived attempts of an unmapped task instance.
Once it is promoted or removed they have no current attempt, and a
cleared mapped task would make every later downgrade refuse. The delete is
scoped to the placeholder's own region so expanding one loop pass cannot
remove a sibling pass's archived attempts. It discards real earlier runs
of a task that later became mapped.
@ashb
ashb force-pushed the worktree-migration0142-downgrade branch from ad93457 to a065aec Compare October 15, 2026 09:48
@ashb
ashb merged commit a87c6dd into apache:main Oct 15, 2026
79 checks passed
@ashb
ashb deleted the worktree-migration0142-downgrade branch October 15, 2026 10:43
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-4-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
❌ v3-4-test Commit Link

You can attempt to backport this manually by running:

cherry_picker a87c6dd v3-4-test

This should apply the commit to the v3-4-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

ashb added a commit that referenced this pull request Oct 15, 2026
…ta is present (#74407)

The 0142 downgrade refused whenever xcom_v2 or rtif_v2 held rows or any
task had retried since the upgrade, so a rollback after any real use of
3.4 was impossible. It now folds that data back into the legacy tables as
if 3.3 had written it. That is only lossless for the latest attempt: the
legacy tables are keyed by task coordinates and can hold values for one
attempt, so earlier attempts' XCom and rendered fields are dropped while
the attempts themselves move back to task_instance_history. The release
note records that cost and the runtime, which grows with the size of xcom
and rendered_task_instance_fields.

Data 3.3 cannot represent (several attempts sharing coordinates, archived
attempts with no current attempt, owners that moved while legacy rows
exist) is refused by read-only checks before the first write, because
MySQL cannot roll back DDL. Past that point a failure on MySQL leaves the
schema partly downgraded and needs a restore from backup.

The downgrade leaves try_number alone. Migration 0138's downgrade already
subtracts one from pending retries and runs after this one on any
downgrade to 3.3, so rewinding here as well made a retried task reuse an
archived try number. Only a test that downgrades through 0138 can see it.

Expansion now deletes the archived attempts of an unmapped task instance.
Once it is promoted or removed they have no current attempt, and a
cleared mapped task would make every later downgrade refuse. The delete is
scoped to the placeholder's own region so expanding one loop pass cannot
remove a sibling pass's archived attempts. It discards real earlier runs
of a task that later became mapped.

(cherry picked from commit a87c6dd)
@ashb

ashb commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

Pushed to 2af7b35 on v3-4-test.

vatsrahul1001 pushed a commit that referenced this pull request Oct 20, 2026
)

test_promotion_leaves_earlier_placeholder_task_instances_at_their_coordinates
asserted that the archived -1 attempt of a mapped task survives expansion.
Since #74407 expansion deletes those attempts, but the delete runs with
synchronize_session=False, so session.get() kept returning the stale
identity-map object and the test passed while asserting the opposite of
what the database holds.

Expire the session and query the database, asserting the archived
placeholder attempt is gone, and rename the test to match.
vatsrahul1001 pushed a commit that referenced this pull request Oct 20, 2026
…on test (#75175) (#75181)

test_promotion_leaves_earlier_placeholder_task_instances_at_their_coordinates
asserted that the archived -1 attempt of a mapped task survives expansion.
Since #74407 expansion deletes those attempts, but the delete runs with
synchronize_session=False, so session.get() kept returning the stale
identity-map object and the test passed while asserting the opposite of
what the database holds.

Expire the session and query the database, asserting the archived
placeholder attempt is gone, and rename the test to match.
(cherry picked from commit 11ed204)

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>
vatsrahul1001 pushed a commit that referenced this pull request Oct 20, 2026
…ta is present (#74407)

The 0142 downgrade refused whenever xcom_v2 or rtif_v2 held rows or any
task had retried since the upgrade, so a rollback after any real use of
3.4 was impossible. It now folds that data back into the legacy tables as
if 3.3 had written it. That is only lossless for the latest attempt: the
legacy tables are keyed by task coordinates and can hold values for one
attempt, so earlier attempts' XCom and rendered fields are dropped while
the attempts themselves move back to task_instance_history. The release
note records that cost and the runtime, which grows with the size of xcom
and rendered_task_instance_fields.

Data 3.3 cannot represent (several attempts sharing coordinates, archived
attempts with no current attempt, owners that moved while legacy rows
exist) is refused by read-only checks before the first write, because
MySQL cannot roll back DDL. Past that point a failure on MySQL leaves the
schema partly downgraded and needs a restore from backup.

The downgrade leaves try_number alone. Migration 0138's downgrade already
subtracts one from pending retries and runs after this one on any
downgrade to 3.3, so rewinding here as well made a retried task reuse an
archived try number. Only a test that downgrades through 0138 can see it.

Expansion now deletes the archived attempts of an unmapped task instance.
Once it is promoted or removed they have no current attempt, and a
cleared mapped task would make every later downgrade refuse. The delete is
scoped to the placeholder's own region so expanding one loop pass cannot
remove a sibling pass's archived attempts. It discards real earlier runs
of a task that later became mapped.

(cherry picked from commit a87c6dd)
vatsrahul1001 pushed a commit that referenced this pull request Oct 20, 2026
…on test (#75175) (#75181)

test_promotion_leaves_earlier_placeholder_task_instances_at_their_coordinates
asserted that the archived -1 attempt of a mapped task survives expansion.
Since #74407 expansion deletes those attempts, but the delete runs with
synchronize_session=False, so session.get() kept returning the stale
identity-map object and the test passed while asserting the opposite of
what the database holds.

Expire the session and query the database, asserting the archived
placeholder attempt is gone, and rename the test to match.
(cherry picked from commit 11ed204)

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db-migrations PRs with DB migration backport-to-v3-4-test Backport to v3-4-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants