Hacker News Reader

Comment by shellfishgene | Hacker News Reader

What I need is a du that caches the results somewhere and then does not rescan the 90% of dirs that have not changed when I run it again a month later...

View on HN

And it would know they did not change without scanning them because how?

It could hash the contents of a dir. Along the lines of git

Except hashing requires... reading.

There is not much to be done here. Directories entries are just names, no guarantees that the files were not modified or replaced.

The best you could do is something similar to the strategies of rsync, rely on metadata (modified date, etc) and cross fingers nobody did `cp -a`.

I would be fine with the latter, the program could display a warning like "Results may be inaccurate, full scan required" or something.

I guess I'm just annoyed that for Windows/NTFS really fast programs are available but not for Linux filesystems.

And to hash something needs reading all of its data. I think deducing the file size would actually be faster in some file systems and never slower with any.

Faster in all file systems I'd guess, stat is fast, opening the file and reading its contents and updating a checksum is slow, and gets slower the larger the file is.

Maybe it could run in the background and use inotify to just update the database all the time, or at least keep track of what needs rescanning?

Thinking about this some more, does this system not already exist for the disk quota calculation in the kernel? How does that work? Would it be possible for a tool to scan the disk once, and then get information about file modifications from the system that's used to update quota info?

Reply on news.ycombinator.com