Repository navigation
Fix Dag callbacks dropped when bundle versioning is disabled - #72930
Merged
ephraimbuddy merged 3 commits intoSep 18, 2026
Merged
Conversation
A Dag run that is not pinned to a bundle version -- the normal case under disable_bundle_versioning -- produces a callback request that carries no bundle version. Callback queueing built a second, uninitialized bundle instance for it and read a path off it, so for a versioned bundle whose path is only defined after initialization the callback was queued against an unresolved location and lost. Nothing surfaced to the user, because a callback-only run records no import error. Such a request wants the bundle's current contents, which is what this processor's own instance already tracks, so serving it from there resolves the same path the file scan queues and keeps a fetch and checkout out of the parsing loop. Nothing else is a valid answer for an unversioned request: callbacks are claimed only for bundles this processor parses, and a versioned instance that is still uninitialized has just failed initialization in the refresh that precedes callback fetching, which is also where it is retried. closes: apache#64891
An initialization outage must not consume pending callbacks before the processor can execute them. Callback work needs to survive temporary bundle failures and remain diagnosable until recovery.
uranusjr
approved these changes
Sep 18, 2026
ephraimbuddy
commented
Sep 18, 2026
Contributor
Backport failed to create: v3-3-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker b6d67b7 v3-3-testThis should apply the commit to the v3-3-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
This was referenced Sep 21, 2026
1 of 2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A Dag run that is not pinned to a bundle version -- the normal case under
disable_bundle_versioning -- produces a callback request that carries no
bundle version. Callback queueing built a second, uninitialized bundle
instance for it and read a path off it, so for a versioned bundle whose path
is only defined after initialization the callback was queued against an
unresolved location and lost. Nothing surfaced to the user, because a
callback-only run records no import error.
Such a request wants the bundle's current contents, which is what this
processor's own instance already tracks, so serving it from there resolves the
same path the file scan queues and keeps a fetch and checkout out of the
parsing loop. Nothing else is a valid answer for an unversioned request:
callbacks are claimed only for bundles this processor parses, and a versioned
instance that is still uninitialized has just failed initialization in the
refresh that precedes callback fetching, which is also where it is retried.
closes: #64891
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Opus 5, GPT-6 astra following the guidelines