Hacker News Reader

Comment by jxramos | Hacker News Reader

do you know of any tools for consistency/accuracy? I've been meaning to do some basic md5 checks for all my files but haven't gotten around to it. Every now and then I find a corrupt jpg image that has a rectangular band running through it or something of the like.

View on HN

Generally I'd recommend using a filesystem designed for that. Something like btrfs or (I think) ZFS, which have checksums built into the filesystem (and if you set them up in a RAID configuration, these checksums can be used to correct data as well)

I believe that's some form of bitrot. AFAIK only ZFS can deal with this (assuming you use ECC RAM).

EDIT: According to Wikipedia, ZFS, Btrfs and ReFS offer strategies to deal with various forms of data degradation, although it appears as if ZFS is still king.

https://en.wikipedia.org/wiki/Data_degradation

For all my important photos I use par files. Using command-line tools like par2create and par2repair is simply part of the routine of storing photos from the camera on the NAS.

I also use it for music files, in the past too many mp3s got broken.

Yup, I also do this. Photos get moved into monthly folders, and every now and then, I run "par2create par2file *".

Before I did this, in the past, I restored from backup and found I had damaged jpeg files. With par2, there's at least a recourse.

parchive and md5deep

http://md5deep.sourceforge.net/

Reply on news.ycombinator.com