Recursive batch md5sum

The md5sum program is used to generate MD5 hashes of files. These can be used later to determine if the file has been modified or corrupted. I usually generate md5sums for all my backed up files and check the sums occasionally. If I find any corrupted files I can restore them from other backups. It's easiest to run md5sum on an entire directory recursively so that all files have an md5sum produced.

Generating md5sums recursively

To run md5sum recursively for the current directory, run the following command from a Bash shell:

find . -type f -exec md5sum  {} \;

The find command finds all files (-type f) in the current directory recursively and then executes md5sum for each of them. To only process files in the current directory, use -maxdepth 1 with find. You can change the value of -maxdepth as needed to include more levels of subdirectories. Omitting it will recurse through all levels of subdirectories.

You can save the MD5 results by redirecting the output to a file:

find . -type f -exec md5sum  {} \; > md5sums.txt

When output is redirected, an empty md5sums.txt file is created first. The find command is then run for each file, including the md5sums.txt file that is in the directory. The MD5 hash for that file is going to be wrong because the file is still being generated when the hash for it is calculated. Therefore, you should edit the md5sums.txt file and remove the entry for itself. You can use a grep command for that to save time, e.g.:

grep -v \.\/md5sums\.txt md5sums.txt > md5sums_fixed.txt

Checking md5sums

The hashes of files can be checked by the following:

md5sum -c md5sums.txt

Discussion

Ano-Nimmis, 2016/12/23 12:03
Thanks for sharing, I love little one liners that just work.
I would love to hear your feedback. Enter your comment below [ Terms of Use ]:
AROIF
 

About Peter Yu I am a research and development professional with expertise in the areas of image processing, remote sensing and computer vision. I received BASc and MASc degrees in Systems Design Engineering at the University of Waterloo. My working experience covers industries ranging from district energy to medical imaging to cinematic visual effects. I like to dabble in 3D artwork, I enjoy cycling recreationally and I am interested in sustainable technology. More about me...

Feel free to contact me with any questions about this site at [user]@[host] where [user]=web and [host]=peteryu.ca

Copyright © 1997 - 2019 Peter Yu