Repository navigation
Error publishing datafusion-spark in datafusion 55.0.0 release "cannot find module or crate datafusion in this scope" #24474
Description
Describe the bug
To Reproduce
Run
(cd datafusion/spark && cargo publish)This errored like
Compiling datafusion-spark v55.0.0 (/Users/andrewlamb/Downloads/apache-datafusion-55.0.0/target/package/datafusion-spark-55.0.0)
error[E0433]: cannot find module or crate `datafusion` in this scope
--> src/function/string/quote.rs:20:5
|
20 | use datafusion::logical_expr::{Coercion, ColumnarValue, Signature, TypeSignatureClass};
| ^^^^^^^^^^ use of unresolved module or unlinked crate `datafusion`
|
= help: if you wanted to use a crate named `datafusion`, use `cargo add datafusion` to add it to your `Cargo.toml`
For more information about this error, try `rustc --explain E0433`.
error: could not compile `datafusion-spark` (lib) due to 1 previous error
error: failed to verify package tarball
I had to change it to refer to datafusion_expr to publish it:
- use datafusion::logical_expr::{Coercion, ColumnarValue, Signature, TypeSignatureClass};
+ use datafusion_expr::{Coercion, ColumnarValue, Signature, TypeSignatureClass};Expected behavior
Should publish cleanly
Additional context
I think the issue is that datafusion is an optional dependency for datafusion-spark which is activated in the workspace
We can probably catch this error in CI by doing clippy check on datafusion-spark without default features
Activity
ci: Check datafusion-spark feature combinations to catch publish erro…
[branch-55] Reduce datafusion-spark dependency footprint (#24351) (#2…