Generalize SQL pushdown to Iceberg/Unity Catalog attached databases - #996
Merged
adsharma merged 1 commit intoSep 19, 2026
Merged
Conversation
ForeignJoinPushDownOptimizer no longer assumes DuckDB-file-catalog shapes: - Column lookup passes the qualified catalog.schema.table reference through to the attached database (which scopes information_schema when it can), with an unqualified fallback so attached databases from older extension builds keep working. - Rel-table join columns prefer the src/dst naming convention (the same convention createForeignRelTable uses) instead of blindly taking the first two columns, falling back to ordinal position. SQL catalogs such as Iceberg REST or Unity Catalog do not guarantee endpoint columns come first. - ATTACH error hint now also suggests loading the iceberg/uc_catalog extensions. Backward compatible in both directions: works with old extension builds (unqualified fallback) and old core builds are unaffected by the extension-side changes. Companion extensions PR: LadybugDB/extensions#86.
adsharma
force-pushed
the
feat/sql-pushdown-for-iceberg-unity
branch
from
September 19, 2026 20:06
2ac3195 to
b457d97
Compare
adsharma
added a commit
that referenced
this pull request
Sep 20, 2026
- Resolve the node-table ID column from the bound catalog entry's primary key instead of assuming it is the first foreign column; fall back to an `id`-named column before ordinal position. - Simplify the src/dst endpoint predicate into an isEndpointColumn helper shared (by convention) with DuckDBCatalog. - Document the all-or-nothing endpoint fallback and the unqualified table-name fallback trade-off; extract quote-stripping helpers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generalizes the existing SQL pushdown so it works with Iceberg REST and Unity Catalog attached databases, not just DuckDB/Postgres/SQLite file catalogs.
Changes
ForeignJoinPushDownOptimizerpasses the qualifiedcatalog.schema.tablereference through toAttachedDatabase::getTableColumnNames()(which scopesinformation_schemawhen supported), with an unqualified fallback so attached databases from older extension builds keep working.src/dstnaming convention (the same conventioncreateForeignRelTableuses for FK detection) instead of blindly taking the first two columns, with ordinal-position fallback. SQL catalogs such as Iceberg REST or Unity Catalog do not guarantee endpoint columns come first.ATTACH"did you forget to load…" hint now also coversiceberg/uc_catalogdatabase types.Compatibility
Backward compatible in both directions and independently mergeable: new core works with old extension builds (fallback path; verified against the DuckDB/SQLite e2e suites), and old core is unaffected by the extension-side changes.
Companion extensions PR: LadybugDB/extensions#86 (Iceberg ATTACH support, Unity alias/install fixes, shared-catalog hardening). Verified together: join pushdown over an attached DuckDB catalog produces correct pushed SQL with name-detected join keys, and all runnable iceberg/duckdb/sqlite e2e suites pass.