Repository navigation
Keep operator extra links viewable for previous task tries - #73315
amoghrajesh wants to merge 3 commits into
Conversation
|
@ashb would you mind taking a look at this? |
| @staticmethod | ||
| def _read_value(session: Session, key: str, ti_key: TaskInstanceKey) -> Row | None: | ||
| return session.execute( | ||
| XComModel.get_many( |
There was a problem hiding this comment.
There is no get_one on XComModel, although its present in the docstring 🤣. Removing it while I am at it
There was a problem hiding this comment.
| if is_link_xcom_key(x, link_xcom_keys): | ||
| # skip clearing this key as it is an operator link | ||
| continue |
There was a problem hiding this comment.
This should be done on the server side, not the client side I feel (i.e. server shouldn't tell client to clear these)
There was a problem hiding this comment.
Nice, thanks. Moved it to the server. ti_run now resolves the task's link xcom_keys from the cached serdag and leaves them out of xcom_keys_to_clear, so the client just deletes what it is given.
There was a problem hiding this comment.
| ) | ||
| assert mock_xcom_set.mock_calls == [ | ||
| call( | ||
| key="_link_AirflowLink__try_1", |
There was a problem hiding this comment.
Does the storage tab in the UI already filter out the se ^_link_AirflowLink items?
There was a problem hiding this comment.
No, the storage tab doesn't filter these.
Which did you have in mind -
hide all _link_* rows, or only the new _link_*__try_N ones?
The narrower rule keeps the tab identical to today for existing runs...
| .limit(1) | ||
| ).first() | ||
| # Runs from before per-try keys existed only have the unsuffixed key. | ||
| result = self._read_value( |
There was a problem hiding this comment.
Since #74222 landed on main, I think most of this is already handled there. Each retry now runs under a fresh TI UUID and ti_run only clears XComs that the current attempt produced, so an earlier try's link rows aren't touched. On the read side, XComOperatorLink.get_link already filters on try_number=ti_key.try_number.
Is anything left here once you rebase? This branch conflicts with main now. The bit I couldn't work out is the unsuffixed fallback for rows written before the upgrade; that would depend on how #74222 maps xcom_v1 rows to a try number.
|
Ping for this, do we still need this? Woud be great to have that links working again for 3.4.0 |
Fixes #71471
Was generative AI tooling used to co-author this PR?
What
Operator extra links are stored in xcom under one key per task instance, and every retry deletes the task's xcoms before running. An extra link records what one attempt did, so the next attempt destroying it means earlier tries lose their link.
The extra links endpoint and the UI already pass
try_numberend to end, butXComOperatorLink.get_linkignored it and read the single surviving row, so the API looked try-aware while silently serving the latest try's link for every try.Current behaviour
A task that fails once and succeeds on retry leaves a single xcom row holding the second attempt's link. Opening try 1 in the UI shows try 2's link, with nothing indicating it belongs to a different attempt. If the retry dies before
finalize()runs, no link exists at all.This predates Airflow 3. Airflow 2 links read XCom for the runtime IDs they needed
(job id, run id, cluster name)and lost them to the same issue. The rewrite toXComOperatorLinkcarried the gap forward rather than introducing it, but also linked a single finished string written at a known point, which is what makes a per-try key cheap now.Proposed change
Each attempt writes its link under a key of its own,
_link_<LinkClass>__try_<n>, and the retry wipe leaves operator link keys alone. The resolver looks up the key for the try being viewed and falls back to the unsuffixed key when there is no per try row.Changes of Note
Filtering happens on the worker, not the API server. The server lists which xcom keys exist; the worker already holds the task object and knows its link keys, so no Dag parsing is added to the
ti_runpath and the execution API contract is untouched.The legacy row is kept, not wiped. After upgrading, a row written before this change survives and acts as the fallback for tries that have no per try row. Old tries keep showing what they show today rather than losing their button.
User implications / backcompat
No schema change, no migration, no config.
Testing
Unit tests cover the resolver reading the requested try, the fallback to the unsuffixed key, the per try write in
finalize(), and the wipe skipping link keys while still clearing everything else. Each fails without the source change.Manual validation
Using this DAG:
Simple situation: Clean database
Try 1:
Try 2:
Situation where old format was in xcom already: newly upgraded
Try 1 and 2 were already run so they both show older behaviour: showing link with
try=2On upgrade to this new version, the task is cleared and third try is launched and it has link with
try=3:Upgrade and mixed version cases checked by hand:
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.