Repository navigation
Add type stubs for Cython classes in pywt._extensions #846
Description
Activity
Hi @NiklasMueller2001, thanks for suggesting this, that seems like a good thing to add indeed. This may be a bit of a project given the size of the API and the support for fused types (32/64-bit float and complex dtypes) in the core functions. It might fit in one large PR, or be broken into multiple PRs - whatever seems best to you.
Ideally it'd come with a CI job to verify that the added stubs are correct and complete. Given
pyrightis now unmaintained andmypyis slow and under-maintained, I think I'd prefer usingpyrefly(it has a coverage report now: https://pyrefly.org/en/docs/report/).I would preferably create one large PR and add commits incrementally to it. Just to clarify the scope of this project, when looking through the pywt._extensions module I identified the following functions/classes that could use some stub files:
_cwt.pyx:- cwt_psi_single
_dwt.pyx:- dwt_max_level
- dwt_coeff_len
- dwt_single
- idwt_single
- dwt_axis
- idwt_axis
- upcoef
- downcoef
_pywt.pyx:- wavelist
- families
- DiscreteContinuousWavelet
- Wavelet
- ContinuousWavelet
_swt.pyx:- swt_max_level
- swt
- swt_axis
That plan sounds good to me. Those are the most important functions indeed. There is more in pure Python code though that is also untyped (everything in https://pywavelets.readthedocs.io/en/latest/ref/index.html). Perhaps that can be inferred easily though, e.g. with https://pyrefly.org/en/docs/stubgen/. It can also be left for later if the scope of that is too much.
- Reacted by Joren Hammudoglu and Niklas Müller
Interestingly, the API of public functions seem to be correctly propagated even with the current dynamic version with wildcard imports like this:
Lines 14 to 15 in 45f51e3
from ._functions import * from ._multilevel import * When import from
pywtI can see the signature of public functions correctly:import pywt pywt.central_frequency # type checking works for public function pywt._integrate # private functions are not known
I opened a draft PR now (#847) that already implements the stub files and removes wildcard imports in
pywt/__init__.py. As for the CI job, pyrefly reports over 200 errors that originate from core python modules independent of the untyped Cython modules. So how should we handle this? One idea would be to fix the current state of the pyrefly report using pyrefly baselines and let the job fail whenever a new type error is introduced.Reacted by Ralf GommersOne idea would be to fix the current state of the pyrefly report using pyrefly baselines and let the job fail whenever a new type error is introduced.
I'm fine with failing on new type errors in CI. For the Pyrefly baselines idea: looks reasonable at first sight, if we can't fix those errors up. NumPy and SciPy are fully typed though, and I wouldn't expect to see typing errors from stdlib usage. So let's see first in how far we can get with fixing up those errors?
Reacted by Niklas MüllerYeah, I will give Pyrefly's stubgen a shot. Can I add those type hinting additions to pure python modules also to the same PR?
Hi, I added a lot of type annotation to the pure python files to achieve better coverage and created a CLI workflow to check coverage regressions. Unfortunately, pyrefly still reports hundreds of type errors inside quite convoluted functions, due to imprecise interfaces / complicated branching logic. In total the errors amount to 310. I would find it very tedious to try to manually clean this up without breaking some edge cases. That's why I stuck to type hints and coverage reports in the PR.
Currently, many public classes and functions are implemented in the Cython modules inside
pywt._extensions(e.g.wavelist,Wavelet, ...). Their public API is currently documented in the pywavelet docs; however, when writing Python code that uses these functions or classes Python type checkers such as mypy or Pyright can not determine their signature and return types or even detect their existence.Possible solution
To enable static type checking I propose to add .pyi interface files in the
pywt._extensionsmodule. I have already created a draft version that works in my locally built pywavelets installation. Please let me know if you think this would be a useful enhancement for the package.