You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I propose that we migrate all of the codecs currently defined in the codecs.numcodecs module to stand-alone repositories under the zarr-developers organization. Those repos can themselves depend on numcodecs, or not. We can then decide if these codecs are worth including as optional dependencies for zarr python.
I think we should also look at removing the numcodecs dependency for our core codecs -- gzip is a python built-in, and there are blosc wheels iirc. zstandard is probably available as a backported python library? We should explore this.
those codecs are currently thin wrappers over numcodecs, and those wrappers are effectively bugged, because they don't do any input validation. When we do full, proper zarr v3 support for those codecs, like #3774, the code size might grow considerably.
I'm also motivated to cut down on our numcodecs dependency as much as we can. These both argue for potentially externalizing codecs.
Activity
maxrjones commentedon Mar 17, 2026
this module - https://github.com/zarr-developers/zarr-python/blob/main/src/zarr/codecs/numcodecs/_codecs.py? There's only ~300 lines of code there, so I'm not really getting the rationale.
d-v-b commentedon Mar 18, 2026
those codecs are currently thin wrappers over numcodecs, and those wrappers are effectively bugged, because they don't do any input validation. When we do full, proper zarr v3 support for those codecs, like #3774, the code size might grow considerably.
I'm also motivated to cut down on our numcodecs dependency as much as we can. These both argue for potentially externalizing codecs.