Repository navigation
Bound remaining single-row lookups with .limit(1) - #72699
pierrejeambrun merged 2 commits into
Conversation
28297da to
629035d
Compare
629035d to
4464b4f
Compare
Two more callers of ``session.scalars(...).first()`` / ``session.execute(select(...)).first()`` were still missing an explicit ``.limit(1)``. Because SQLAlchemy 2.0's ``Result.first()`` does not add one (unlike the legacy ``Query.first``), each call sends every matching row to the Python process and discards all but the first. The favorite existence check is bounded to a single row per (dag_id, user_id) by the schema, but the previous-TI lookup scans every earlier matching TI. Introduced independently of apache#52325's migration: get_previous_task_instance came in apache#59712, unfavorite_dag's existence probe in apache#51264. Same anti-pattern as the four sites apache#72554 fixes; the tests reuse the ``capture_orm_selects`` helper that PR introduces.
4464b4f to
9c4a9c2
Compare
9c4a9c2 to
3b62671
Compare
Backport failed to create: v3-3-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker ff83957 v3-3-testThis should apply the commit to the v3-3-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
* Bound remaining single-row lookups with .limit(1) Two more callers of ``session.scalars(...).first()`` / ``session.execute(select(...)).first()`` were still missing an explicit ``.limit(1)``. Because SQLAlchemy 2.0's ``Result.first()`` does not add one (unlike the legacy ``Query.first``), each call sends every matching row to the Python process and discards all but the first. The favorite existence check is bounded to a single row per (dag_id, user_id) by the schema, but the previous-TI lookup scans every earlier matching TI. Introduced independently of #52325's migration: get_previous_task_instance came in #59712, unfavorite_dag's existence probe in #51264. Same anti-pattern as the four sites #72554 fixes; the tests reuse the ``capture_orm_selects`` helper that PR introduces. * Also assert the returned row on the previous-TI regression test (cherry picked from commit ff83957) Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
* Bound remaining single-row lookups with .limit(1) Two more callers of ``session.scalars(...).first()`` / ``session.execute(select(...)).first()`` were still missing an explicit ``.limit(1)``. Because SQLAlchemy 2.0's ``Result.first()`` does not add one (unlike the legacy ``Query.first``), each call sends every matching row to the Python process and discards all but the first. The favorite existence check is bounded to a single row per (dag_id, user_id) by the schema, but the previous-TI lookup scans every earlier matching TI. Introduced independently of apache#52325's migration: get_previous_task_instance came in apache#59712, unfavorite_dag's existence probe in apache#51264. Same anti-pattern as the four sites apache#72554 fixes; the tests reuse the ``capture_orm_selects`` helper that PR introduces. * Also assert the returned row on the previous-TI regression test
|
@pierrejeambrun @jason810496 aren't a good chunk of these better as |
We will need to check the behavior of SQLA2 on the |
IIUC, both |
|
Thanks @FrankYang0529 for the follow-up so we will keep them as-is (keep the |
Yes, I'd keep the |
|
Yep, db scan cost + transfert time. On the db side there is no |
Follow-up to #72554, and depends on it. That PR restores
.limit(1)on thefour single-row XCom lookups whose
LIMITwas dropped when the code migratedfrom the legacy
Query.first()to SQLAlchemy 2.0'sResult.first()(whichdoes not add a LIMIT of its own — see the
Result.firstdocstring). A wideraudit of the remaining
.first()calls underairflow/api_fastapi/turnedup two more sites with the same shape:
execution_api/routes/task_instances.py::get_previous_task_instance— readsone previous TI ordered by
logical_date DESC, no LIMIT. Without the boundthe query pulls every earlier matching TI into the Python process and returns
the first. Introduced in Add ability to get previous TI on
RuntimeTaskInstance#59712.core_api/routes/public/dags.py::unfavorite_dag— probesdag_favoriteforthe current
(dag_id, user_id)before deletion. The schema bounds the resultto a single row, so the practical cost here is small, but the pattern is the
same. Introduced in Add support for favorite/pin dags to dashboard #51264.
The regression tests reuse the
capture_orm_selectshelper #72554 adds todevel-common, so this branch is stacked on top of that PR and should bemerged after it.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.7) following the guidelines