Hacker News Reader

I don't know if it's related, but one time I had corruption too. I was playing with large sparse (ntfs) files and used zstd for temporarily store those. It was probably low compression settings and I couldn't reproduce, so I'd thought I messed up something.

View on HN

oh, I remembered, it was not all zstd fault. Somehow because zstad read not only the allocated space but the whole fole, the zeros too, windows tried to allocate the whole file in memory, which was not enough, and zstd did not give (outofmemory) eror message. (It was not ntfs compressed, only sparse!) Later (maybe windowsupdate happend) tried to reproduce, but worked with larger file with memory.

I don't know zstd how could handle the outofmemory, but the sparse reading might not be perfect.

Reply on news.ycombinator.com