You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The metric dag.dag_id.task_id.queued_duration is not being emitted after Airflow 3 upgrade. There are multiple issues like this where metrics are reported missing (we have also noticed ti.finish, ti.start missing, but that has been fixed in 3.1.8).
I had a look at this one. The root cause is that queued_duration is emitted by TaskInstance.emit_state_change_metric(RUNNING), which in Airflow 2 ran when the task transitioned to RUNNING. In Airflow 3 that transition moved to the execution API (the ti_run route), so the call site in taskinstance.py no longer fires and the metric is never emitted.
Its sibling scheduled_duration still works because it is emitted on the QUEUED transition, which is still triggered server side from the scheduler (scheduler_job_runner).
The change that keeps the two symmetric is to emit task.queued_duration server side in the ti_run handler when a task goes from QUEUED to RUNNING, computing now - queued_dttm. The metric template already maps task.queued_duration to the legacy dag.<dag_id>.<task_id>.queued_duration name, so both the tagged and statsd forms are covered.
@seanmuth are you planning to take this one? If not, I am happy to open a PR with a test.
Activity
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
fix(metrics): emit queued_duration metric in execution API ti_run end…
17 remaining items
eladkal commentedon May 26, 2026
@seanmuth you previously fixed some of the missing metrics. Will you have time to look into this one?
wilmerdooley commentedon May 28, 2026
I had a look at this one. The root cause is that
queued_durationis emitted byTaskInstance.emit_state_change_metric(RUNNING), which in Airflow 2 ran when the task transitioned to RUNNING. In Airflow 3 that transition moved to the execution API (theti_runroute), so the call site intaskinstance.pyno longer fires and the metric is never emitted.Its sibling
scheduled_durationstill works because it is emitted on the QUEUED transition, which is still triggered server side from the scheduler (scheduler_job_runner).The change that keeps the two symmetric is to emit
task.queued_durationserver side in theti_runhandler when a task goes from QUEUED to RUNNING, computingnow - queued_dttm. The metric template already mapstask.queued_durationto the legacydag.<dag_id>.<task_id>.queued_durationname, so both the tagged and statsd forms are covered.@seanmuth are you planning to take this one? If not, I am happy to open a PR with a test.
mahesh-github commentedon May 28, 2026
Can I take this one pls
Restore queued_duration metric in Airflow 3
Restore queued_duration metric in Airflow 3
Fix missing task.queued_duration metric in Airflow 3
Fix missing task.queued_duration metric in Airflow 3
Fix missing task.queued_duration metric in Airflow 3
Fix missing task.queued_duration metric in Airflow 3 (#67592)
[v3-4-test] Fix missing task.queued_duration metric in Airflow 3 (#67592
[v3-4-test] Fix missing task.queued_duration metric in Airflow 3 (#67592