Show HN: Stream-unzip – stream unZIP files in Python

Last I was dealing with zip files in Python 3 years ago, I was unpleasantly surprised that there is no way to open Zip64 archives.

It is a semi-proprietary format not supported by zlib. Some tools, most notably Windows will sometimes use it to create large zip archives that go beyond limits of original zlib. An unsuspecting user would compress some files and there is no warning which format has been used.

would have been useful for something we ran into at a previous job. For legacy reasons large csv files had been zipped instead of gzipped (or some other streamable format), and were causing some process in our data pipeline to consume too much memory.

I wound up coming up with a quick solution that appeared to work. Taking advantage of being able to get s3 objects at arbitrary ranges, I implemented seek and read on top.