1) either mount tar directly
2) or use squashfs and mount that...both would be mounted with some sort of rw overlay
3) underneath do some lazy httpfs type filesystem, so filesystem can be mounted while it's downloading
4) parallelize the underlying download ala aria2c
5) provide metadata to aria2c-alike downloader with file boundaries, so extract further latency/bandwidth savings
2) Agreed, still need to look more into this, although it's more involved and entails changing more of our pipeline for packaging and distributing these images.
3) I need to look into how this would handle files that haven't been downloaded yet. From what I know about httpfs filesystems, I'm not sure how much would need to be done to let this block until file needed is downloaded vs the normal behavior of calling out to get the file being requested.
5) Could you go into more detail here? Not sure I understand how file boundaries, etc can help with latency/bandwidth.
6)I forgot to do the best part of the optimization..do feedback-guided-optimization. You can then repack the squashfs/tar file to have all the frequently-accessed files together