Skip to content

Fix mapped-task retry policy example in tasks docs - #74343

Merged
kaxil merged 1 commit into
apache:mainfrom
astronomer:fix-mapped-retry-policy-docs-example
Oct 6, 2026
Merged

kaxil merged 1 commit into
apache:mainfrom
astronomer:fix-mapped-retry-policy-docs-example

Conversation

@kaxil

@kaxil kaxil commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

The "Mapped tasks" example under retry policies in the tasks docs does not run:

@task.partial(retry_policy=my_policy).expand(input=[1, 2, 3])
def my_mapped_task(input): ...

task is a TaskDecoratorCollection, so task.partial is looked up as a provider task decorator and raises AttributeError: task decorator 'partial' not found. Calling .expand() as a decorator is not a supported pattern either. The example came in with #73027.

This replaces it with the two forms that work: @task(retry_policy=...) followed by .expand(), and .partial(retry_policy=...) on a classic operator. The surrounding prose said the policy is set "via .partial()", which only holds for the classic form, so it now names both.

Before

A Dag file containing the documented snippet fails to import:

Dag import error for the documented snippet: AttributeError: task decorator 'partial' not found

After

Both new snippets, run in Airflow with a policy that returns FAIL on ValueError and a task with retries=3 that raises it for input 2. The policy fails map index 1 on its first try, and the other two instances succeed:

Mapped instances of my_mapped_task: map index 1 failed on try 1, 0 and 2 succeeded

The log of map index 1 shows the policy decision:

Task log of map index 1 showing "Retry policy decision action=fail"

The classic BashOperator.partial(task_id=..., retry_policy=...).expand(...) form maps three instances:

Mapped instances of the BashOperator task, all three succeeded

Dag used for the "after" run
from datetime import timedelta

from airflow.providers.standard.operators.bash import BashOperator
from airflow.sdk import DAG, ExceptionRetryPolicy, RetryAction, RetryRule, task

my_policy = ExceptionRetryPolicy(
    rules=[RetryRule(exception=ValueError, action=RetryAction.FAIL, reason="Bad input, retrying won't help")]
)

with DAG(dag_id="retry_policy_mapped_after", schedule=None):

    @task(retry_policy=my_policy, retries=3, retry_delay=timedelta(seconds=1))
    def my_mapped_task(input):
        if input == 2:
            raise ValueError(f"bad input: {input}")
        return input

    my_mapped_task.expand(input=[1, 2, 3])

    BashOperator.partial(task_id="run", retry_policy=my_policy).expand(
        bash_command=["echo 1", "echo 2", "echo 3"],
    )

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

The example used `@task.partial(...).expand(...)` as a decorator, which
raises `AttributeError: task decorator 'partial' not found`. Show the
`@task(retry_policy=...)` + `.expand()` form and the classic operator
`.partial(retry_policy=...)` form instead.
@kaxil
kaxil merged commit de420be into apache:main Oct 6, 2026
71 checks passed
@kaxil
kaxil deleted the fix-mapped-retry-policy-docs-example branch October 6, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants