Replace existing duplicate files on APFS with clones

Solution 1:

As of June 2020, Jdupes can do this.

For example, this will find all duplicates in your Applications folder:

jdupes --recurse --dedupe /Applications/ 

WARNING: Versions before 1.19.0 will overwrite metadata (date, owner, permission), more recent versions preserve all metadata when using jdupes --dedupe on APFS filesystems.

Solution 2:

I had the same problem and wrote a small shell script to do this using native cp, mv, jdupes (for speed), and gcp (for metadata preservation). Hope this helps!

Advantage over other solutions:

• http://diskdedupe.com and https://macpaw.com/gemini: Free, open-source, use byte-wise comparison to prevent collisions
• https://github.com/deckarep/apfs-compactor & https://bitbucket.org/dchevell/apfs-deduplicate: Preserve metadata, use byte-wise comparison to prevent collisions
• https://github.com/ranvel/clonefile-dedup: No SQL/Python dependencies, use byte-wise comparison to prevent collisions