Repository navigation
ci: remove lru from deny list - #9896
Merged
westonpace merged 4 commits intoOct 12, 2026
Merged
Conversation
Contributor
|
ACTION NEEDED The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. For details on the error please inspect the "PR Title Check" action. |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Contributor
There was a problem hiding this comment.
The lru exception remains obsolete: all three lockfiles use patched version 0.16.4. The added exception is narrowly scoped to an informational maintenance advisory, so no change is required before merge. It can be retired when the YAML dependency paths migrate to maintained libraries.
| { id = "RUSTSEC-2026-0194", reason = "`quick-xml` <0.41 pulled transitively via object_store (datafusion) and reqsign-aws-v4 (opendal); upstream must upgrade first" }, | ||
| { id = "RUSTSEC-2026-0195", reason = "`quick-xml` <0.41 pulled transitively via object_store (datafusion) and reqsign-aws-v4 (opendal); upstream must upgrade first" }, | ||
| { id = "RUSTSEC-2026-0002", reason = "`lru` 0.12.x pulled transitively via goosefs-sdk; fixed in >=0.16.3, goosefs-sdk must upgrade first" }, | ||
| { id = "RUSTSEC-2024-0449", reason = "`serde_yaml` pulled transitively via datafusion-substrait; serde features not utilized by us" }, |
Contributor
There was a problem hiding this comment.
The new advisory concerns the archived unsafe-libyaml crate and lists no patched release. The lockfiles include it through both Substrait’s serde_yaml and Lindera’s serde_yaml_ng; Python also calls serde_yaml::to_string in LanceFileMetadata.__repr__. The waiver therefore accepts an ongoing maintenance risk in YAML code that Lance uses, beyond the unused Substrait serde features mentioned in the reason. The advisory reports no specific vulnerability, so this is non-blocking.
Contributor
There was a problem hiding this comment.
The Python formatter now uses yaml_serde/libyaml-rs in c25c855, removing its direct unsafe-libyaml dependency. The remaining non-blocking maintenance risk comes from Substrait's build dependency and Lindera's existing YAML dependency.
Contributor
There was a problem hiding this comment.
The Python formatter now uses yaml_serde/libyaml-rs, and the focused metadata serialization checks preserve the previous YAML output. All three lockfiles still use patched lru 0.16.4. The remaining maintenance risk is in the existing Substrait/Lindera YAML dependencies; the targeted exception remains reasonable until those dependencies migrate.
LuQQiu
approved these changes
Oct 12, 2026
amunra
added a commit
to rerun-io/lance
that referenced
this pull request
Oct 20, 2026
* ci: remove lru from deny list (lance-format#9896) It appears this deny entry is no longer required. Also adds serde_yaml which is used transitively through `datafusion-substrait` and `lindera` though we do not exercise these code paths. We also had one direct dependency on serde_yaml and this PR migrates this to yaml_serde which is the replacement. (cherry picked from commit f88c716) * ci: update transitive chacha to 0.10.2, ignore rustls CVEs on 0.23.45 (lance-format#9985) The old version had been yanked for UB (likely a crash) on older hardware and was triggering the deny check to fail. This also adds some new ignores for CVEs reported against rustls version 0.23.45. These CVEs are not critical enough to force an upgrade to the newer version since our policy requires 48 hours. (cherry picked from commit 173ab9f) * ci: update yanked spin and chacha20 lockfile entries spin 0.9.8 and 0.10.0 are yanked; upstream main locks 0.9.9 and 0.10.1 via an unrelated change, so this is a `cargo update -p spin` rather than a cherry-pick. The java lockfile is not covered by the cargo-deny job and so was not touched by lance-format#9985; it gets the same chacha20 0.10.2 bump here. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Weston Pace <weston.pace@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
ilya-zlobintsev
pushed a commit
to rerun-io/lance
that referenced
this pull request
Oct 20, 2026
* ci: remove lru from deny list (lance-format#9896) It appears this deny entry is no longer required. Also adds serde_yaml which is used transitively through `datafusion-substrait` and `lindera` though we do not exercise these code paths. We also had one direct dependency on serde_yaml and this PR migrates this to yaml_serde which is the replacement. (cherry picked from commit f88c716) * ci: update transitive chacha to 0.10.2, ignore rustls CVEs on 0.23.45 (lance-format#9985) The old version had been yanked for UB (likely a crash) on older hardware and was triggering the deny check to fail. This also adds some new ignores for CVEs reported against rustls version 0.23.45. These CVEs are not critical enough to force an upgrade to the newer version since our policy requires 48 hours. (cherry picked from commit 173ab9f) * ci: update yanked spin and chacha20 lockfile entries spin 0.9.8 and 0.10.0 are yanked; upstream main locks 0.9.9 and 0.10.1 via an unrelated change, so this is a `cargo update -p spin` rather than a cherry-pick. The java lockfile is not covered by the cargo-deny job and so was not touched by lance-format#9985; it gets the same chacha20 0.10.2 bump here. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Weston Pace <weston.pace@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Xuanwo
pushed a commit
that referenced
this pull request
Oct 23, 2026
…#9827) Deleting a nonempty Lance branch on a TOS bucket with hierarchical namespace can fail with HTTP 409 `CannotDelete`: the OpenDAL bridge exposes directory entries as zero-size objects, and the generic deletion stream may remove a directory before its children. The generic `ObjectStore::remove_dir_all` now defers zero-size entries and deletes them longest-path-first after nonempty files. Zero-byte files are deferred too, which is safe because child paths are longer than their parent directory. Nonempty files are forwarded through one continuous `delete_stream`, retaining each provider's batching and concurrency. All file deletes complete before deferred entries are deleted. Cleanup uses a single wrapped listing, so tracing, metrics, filtering, and path rewriting remain in effect; no native OpenDAL listing or provider directory-operation refactor is needed. Regression tests cover hierarchical and flat TOS listings, pagination with each page read once, mixed nonempty and zero-byte files, nested/empty/missing directories, encoded paths, sibling-prefix isolation, deletion failures, rewritten paths, wrapped stores, and dynamic credentials. A synthetic S3-style backend test verifies that 20,000 nonempty files retain 20 concurrent batches and that deferred entries are deleted only after all file deletes finish. Validation: - `cargo test -p lance-io --features tos object_store::`: 277 passed. - `cargo fmt --all` and `git diff --check`. - `cargo clippy -p lance-io --features tos --tests --benches -- -D warnings`. - `cargo clippy --all --tests --benches -- -D warnings` (Rust 1.98.1). - Reproduced the original failure on upstream `4323cdee1374bfe28971a83f72dddf43f9cbc863` (14.0.0-beta.8). Revalidated the revised fix on a disposable real TOS HNS dataset with two main fragments / four rows and a branch appended to six rows: branch deletion succeeds, its files and directories are removed, and the main dataset still reads all four rows. The disposable dataset was fully cleaned up. This fixes deletion ordering; it does not add rollback of an already removed branch reference if cleanup fails. CI status: - [Lance Gatekeeper recommends approve without changes](#9827 (review)); the empty-commit revision preserves that review. - The earlier transitive `unsafe-libyaml` advisory blocker was addressed upstream in [#9896](#9896). The new CI run on `b31ed5953` passes the dependency audit and license-header check. This PR adds no advisory exception. - See the PR checks for the latest build and test results. --------- Co-authored-by: wangzheyan <wangzheyan@bytedance.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It appears this deny entry is no longer required.
Also adds serde_yaml which is used transitively through
datafusion-substraitandlinderathough we do not exercise these code paths.We also had one direct dependency on serde_yaml and this PR migrates this to yaml_serde which is the replacement.