Find duplicate files on Mac OS X

Solution 1:

I'll just copy part of my other answer on a similar subject and also link to a possible duplicate as another reference.

There are many commercial options, some may be better than the listed below, I haven't tried them all. Anyway, I'm listing my choice of apps considering which ones I was able to try. Coincidentally, this was not where I first heard about DupeCheck droplet, but thanks so much Systech for building it up! :)

• TidyUp is a very well known app in this subject. You can specify where to scan for what kind of duplicates. It offers basic and advanced modes, several different strategies and criterias.

• MrClean is a free tool that just scans for folders for duplicates and trash them. Very simplistic but efficient if you're sure on what you're doing.

• Chipmunk scans duplicates and let you choose which ones you want to trash. It offers a node-view of folders and you can select to "delete all files in a folder that have duplicates elsewhere, or vice versa" as well as hand-picking. It may take very long to scan all files, but it does a very decent job after that.

• DupeCheck "drop a file on it and it will use your Spotlight index to see if you have a potential duplicate somewhere." That's about this nice open source app. Not a great tool for space cleaning at once, but over time it helps you keep your space clean.

• DuplicateFileSearcher from the website: "is a free powerful software utility that will help you to find and delete duplicate files on your computer. It can also be used to calculate MD5 and SHA hashes. The software runs in Windows, Linux, Solaris and MacOS.". Enough said.

Solution 2:

I've converted a bash script into a droplet. Drop a file on it, and it will use your Spotlight index to see if you have a potential duplicate somewhere on your system.
Download the free DupeCheck droplet, unzip it, and find a good spot to put it. Drag and drop any file onto it, and it will use Spotlight to identify possible duplicates.

Solution 3:

I've written a simple python script FileDupeFinder.py that will find file duplicate files on OSX (and probably Linux/*nix) systems. Right now, you pass it a directory root path and a minimum file size and it will walk the directory tree and output a list of files that it believes to be duplicates. It will ignore files that are hard links of each other. You can tell it to exclude directories (right now it just excludes Backups.backupdb, as this script isn't ready to handle Time Machine wackiness) and tell it not to span filesystems (though these aren't currently paramaterized, they are easy enough to set in the script itself.

This script is only minimally tested, but it has worked reasonably enough for me.