If you manage the index using a B-tree, then you can perform partial updates of the B-tree by appending the minimum number of pages needed to represent the changes. At that point, you can append N additional new files to the tail of the archive, and add a new index that re-uses some of the original index.

Just an idea to check the "append" box.

See also B-trees, Shadowing, and Clones https://www.usenix.org/legacy/events/lsf07/tech/rodeh.pdf

Consider variable-length integers for file sizes and string lengths. When I need them, I usually implementing what’s written in MKV spec: https://www.rfc-editor.org/rfc/rfc8794.html#name-variable-si...

That’s a good way to bypass that 4GB size limit, and instead of wasting 8 bytes per number this will even save a few bytes.

However, I’m not sure how easy it is to implement in the language you’re using. I only did that in C++ and modern C#. They both have intrinsics to emit BSWAP instruction (or an ARM equivalent) to flip byte order of an integer, helps with performance of that code.

Don't use 32-bit file times! Change it quick while you have the chance.

How far are we from public beta or Bun 1.0?

Not related to bun or hop is the use of Zig. I am wondering if you will do a write up on it someday.

Have you looked at using LZ4 compression? Supposedly it has a very fast decompressor, so it could be better than uncompressed files for your purposes.

What do you think about Valve VPK?