Unable to resolve the spec apache-airflow[all]

A good requirement to test any resolver against is apache-airflow[all], rip currently throws an error:

$ cargo r -- "apache-airflow[all]"
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/rip 'apache-airflow[all]'`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value:   × Failed parsing requirement string "azure-datalake-store (>=0.0.45azure-mgmt-datalake-store>=0.5.0) ; extra == 'all'")
  ╰─▶ error at 1:56: expected one of "!", ")", "*", "+", ",", "-", ".", "_", letter or digit
', crates/rattler_installs_packages/src/resolve.rs:257:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Thanks! I'll take a look!

We took a closer look at the issue and its because apache-airflow ==1.2.11 contains invalid metadata:

...
Requires-Dist: azure-datalake-store (>=0.0.45azure-mgmt-datalake-store>=0.5.0) ; extra == 'azure_data_lake'
...

We error on this case but pip seems to ignore invalid requirements when resolving:

https://github.com/pypa/pip/blob/2ba5acc8a4772ca1dc0d62a2dfe8967af28649d6/src/pip/_internal/resolution/resolvelib/factory.py#L392

#31 mimics this behavior. However, apache-airflow[all] is still unsolvable because we don't take sdists into account (this is the same output as pip):

|-- apache-airflow 1.10.3 | 1.10.4 | 1.10.5 | 1.10.6 | 1.10.7 | 1.10.8 | 1.10.9 | 1.10.10 | 1.10.12 | 1.10.14 | 1.10.15 | 2.0.0 | 2.0.1 | 2.0.2 | 2.1.0 | 2.1.1 | 2.1.2 | 2.1.3 | 2.1.4 | 2.2.0 |
  │ 2.2.1 | 2.2.2 | 2.2.3 | 2.2.4 | 2.2.5 | 2.3.0 | 2.3.1 | 2.3.2 | 2.3.3 | 2.3.4 | 2.4.0 | 2.4.1 | 2.4.2 | 2.4.3 | 2.5.0 | 2.5.1 | 2.5.2 | 2.5.3 | 2.6.0 | 2.6.1 | 2.6.2 | 2.6.3 | 2.7.0 | 2.7.1 would
  │ require
  │     |-- cron-descriptor >= 1.2.24, for which no candidates were found.

cron-descriptor does not provide any wheels for any matching version.

fix: ignore invalid requirements (#31)

Skips artifacts that have incorrect metadata (#30)

Shouldn't we keep this open until we have some form of sdist support?

I added

Feat: support sdists #32

Navigation Menu

Repository navigation

Unable to resolve the spec `apache-airflow[all]` #27

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Unable to resolve the spec apache-airflow[all] #27

Description

Activity

baszalmstra commented on Sep 28, 2023

baszalmstra commented on Sep 29, 2023

wolfv commented on Sep 29, 2023

baszalmstra commented on Sep 29, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Unable to resolve the spec `apache-airflow[all]` #27