compare two directory trees

I have two music libraries, one a newer version than the other. I would like to compare them to figure which files I need to copy from new music tree to old.

I have tried diff --brief -r /oldmusicdir/ /newmusicdir/ based on another user's suggestion but ^C the process after fifteen minutes (I'm guessing diff is scanning the music files themselves -- which is not necessary).

Then I tried find /oldmusicdir/ -type d | sort > oldmusicdir for old and new, then ran diff oldmusicdir newmusicdir However, since I stored the music directories in separate locations every single entry was flagged.

Next I tried running find /musicdir/ -type d | basename -s - | sort > musicdir but then my musicdir file simply read "-"

Does anyone know how to get basename to accept from STDIN? Or, does anyone have a better way of quickly comparing two music directories?

Thanks!


The rsync utility first popped into my mind when I saw your question. Doing something like below could quickly show what files are in directory a but not in b:

$ rsync -rcnv a/* b/

-r will recurse into the directories
-c will compare based on file checksum
-n will run it as a "dry run" and make no changes, but just print out the files 
   that would be updated
-v will print the output to stdout verbosely

This is a good option because you can compare the contents of the files as well to make sure they match. rsync's delta algorithm is optimized for this type of use case. Then if you want to make b match the contents of a, you can just remove the -n option to perform the actual sync.

Some related questions:

  • https://stackoverflow.com/questions/19396718/compare-files-in-two-directory-on-remote-server-using-unix
  • https://unix.stackexchange.com/questions/57305/rsync-compare-directories

How about generating a recursive directory listing of each directory into separate files and then using diff on those two files?


If you like using some GUI-based utility, you can try some comparison tool: under Windows, my favorite is Total Commander (homepage: https://www.ghisler.com): it has a quite flexible feature that allows to compare and sync entire directory trees; a quick Diff tool is also present, allowing you to inspect the differences encoutered (useful for code lines, less for binary files).

In *ux, I know there's Midnight Commander, but I never tried it and I'm not sure if the same feature is present.

Anyway, Wikipedia has a page where the most used comparison tools are listed: maybe you can find some useful information there: https://en.wikipedia.org/wiki/Comparison_of_file_comparison_tools

Max