Skip to content

Use gzip's GzipFile to manage compression/decompression - #4

Merged
funkey merged 9 commits into
funkey:masterfrom
jakirkham:use_gzipfile_obj
Oct 31, 2018
Merged

funkey merged 9 commits into
funkey:masterfrom
jakirkham:use_gzipfile_obj

Conversation

@jakirkham

@jakirkham jakirkham commented Oct 30, 2018 •

Copy link
Copy Markdown

Related to zarr-developers#87

Makes use of the builtin gzip module's GzipFile backed by a BytesIO buffer to manage compression/decompression. Avoids handling the messy details of the GZip format by letting Python do them for us. Should catch some other format subtleties (e.g. optional headers, footers) as well as handling any checksum issues that might occur.

alimanfoo and others added 5 commits October 30, 2018 14:45
Mirrors variable naming within `encode` in `decode`.
To handle compression and decompression with the GZip format in a Python
2/3 compatible way, use a `GzipFile` instance backed by an in-memory
`BytesIO` buffer for reading and writing from. Offloads the manual
header and footer handling to the Python builtin `gzip` module
simplifying the code a bit.
As the compression/decompression strategy has changed to handle headers
and footers, update the fixtures accordingly.
@jakirkham

Copy link
Copy Markdown
Author

Just as a note, had merged in upstream master above to bring PR ( zarr-developers#87 ) up-to-date along with these changes. So the README changes are already in upstream and can be ignored for the purposes of this PR.

@funkey
funkey merged commit a62fa59 into funkey:master Oct 31, 2018
@jakirkham
jakirkham deleted the use_gzipfile_obj branch October 31, 2018 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants