Skip to content

[AMORO-4420][Hive] Preserve raw Hive partition values when building Iceberg data - #4422

Open
hutiefang76 wants to merge 1 commit into
apache:masterfrom
hutiefang76:frank/oss-hive-partition-20261007-01a11186
Open

hutiefang76 wants to merge 1 commit into
apache:masterfrom
hutiefang76:frank/oss-hive-partition-20261007-01a11186

Conversation

@hutiefang76

Copy link
Copy Markdown
Contributor

Why are the changes needed?

Fixes #4420.

Raw Hive partition values can contain /, % and +. Converting them through a generated partition path confuses value data with path separators and can break metadata synchronization or upgrades.

Brief change log

  • Build PartitionData directly from each raw value using Iceberg's type conversion.
  • Validate the value count against the partition fields.
  • Cover slash-containing strings, verbatim special characters, typed multi-column values and lookup by a real Iceberg partition key.

How was this patch tested?

  • Added deterministic regression tests.
  • Ran tests locally before making this pull request.
  • Screenshots (not applicable).

JDK 17: all 5 TestHivePartitionUtil tests passed.

./mvnw -pl amoro-format-mixed/amoro-mixed-hive -am -Pskip-dashboard-build -Dtest=TestHivePartitionUtil -Dsurefire.failIfNoSpecifiedTests=false test

dev/reformat passed. These are conversion/key-lookup tests; no external Hive metastore or full table-upgrade integration run is claimed.

Documentation

  • Does this pull request introduce a new feature? No.
  • Feature documentation: not applicable; this corrects existing behavior.

AI assistance: DeepSeek and Codex assisted implementation, tests and this description.

@github-actions github-actions Bot added the module:mixed-hive Hive moduel for Mixed Format label Oct 7, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:mixed-hive Hive moduel for Mixed Format

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Hive partition values containing slash fail Iceberg conversion

1 participant