How can I find duplicate photos?

Solution 1:

digiKam

Add all the photos to your collection. In the menu, select Tools / Find duplicates. This will look for duplicates across your whole collection.

findimagedupes

A command line tool. Pass all the images you want to compare on the command line.

Geeqie (formerly GQview)

In the menu, select File / Find duplicate. Drag and drop image files do the duplicates window. You can drop directories to add their contents recursively. For visual comparison of images, there are specific, non-default options on a drop-down menu. The "custom" level of similarity allows restricting pairings only to the highest degree of similarity, but it has to be set on Preferences as 99. Even then, it does not work perfectly at least for some kinds of images, like line-art. It unfortunately does not provide an automatic selection mechanism with rational criteria, such as resolution, date or whatever, the automatic selection seems to just randomly just pick the first image found as the reference to preserve. Deleting many images can be extremely slow, as it tries to update the result count at every delete.

All three of these tools find visual duplicates, not just files that are identical byte for byte.

Solution 2:

FSlint

fslint is a graphical program that can find duplicate files of any type by md5sum. If the images are not identical, they won't be flagged as duplicates. The image below shows a bunch of duplicate pdf files in my Downloads directory:

You can change the advanced search parameters to search by file type and restrict yourself to images only. That's done through changing the "extra find parameters" as find command options. For example, here I am only looking for *.jpg files (in the same path, only looking at my "Downloads" folder:

fdupes

fdupes is an equivalent command-line based tool. Both are available in the repos.