I recommended Zarr on the basis that threading seems to work much better and compression options are better.
I recommended Zarr on the basis that threading seems to work much better and compression options are better.
It's more that I keep hearing that statement ("Zarr is chunked and HDF5 isn't") in the wild a lot.
As for threading, yeah, HDF5 can be a bit cumbersome there.
I'm not sure I'd agree on compression, though. HDF5 supports fully arbitrary compression, after all, it's just the client reading the data also needs to have the compression filter you're using installed. Zarr is often used with BLOSC, and that was originally developed specifically for HDF5, FWIW.
My impression from the talk linked in the article was that HDF5 doesn't do BLOSC by default, and that in general it's much easier to plug new things in, add caching, etc..
pytables might make it possible though, haven't looked.