> >scanning the entire archive would be ridiculously slow on a system with <1MB/s throughput a some kilobytes of RAM

> That's how tar works. Just the other day I was making a backup and streamed 5gb of data as a tar archive, opening it in the archiver took a lot of time.

That's because tar is the tape archiver, specifically written for a medium whose very nature is linear reading/writing and where skipping around was highly impractical or even impossible.