How to clean duplicates existing in one folder from another recursively?
Below are two solutions, depending on how we define "duplicate":
- Files with the same relative path, or
- Files with the same content but not necessarily the same name
If by "duplicate" we mean two files which share the same relative path, then you could use find
and xargs
to remove the duplicates. For example, suppose you have
~/tmp% tree A
A
└── Excellent
├── bar
├── baz
└── foo
~/tmp% tree B
B
├── Bad
│ └── quux
├── Excellent
│ ├── bar
│ ├── baz
│ └── foo
└── Good
Then
find /home/unutbu/tmp/A -depth -type f -print0 | xargs -0 -I{} bash -c 'rm "/home/unutbu/tmp/B${1#*A}"' - {}
results in
~/tmp% tree B
B
├── Bad
│ └── quux
├── Excellent
└── Good
Or, if by "duplicate" we mean two files share the same content, though perhaps not the same filename, then you could use rdfind
:
sudo apt-get install rdfind
If we have this directory structure:
~/tmp% tree A
A
└── Excellent
├── bar
├── baz
└── foo
1 directory, 3 files
~/tmp% tree B
B
├── Bad
│ └── quux
├── Excellent
│ ├── barbar
│ ├── bazbaz
│ └── foofoo
└── Good
where barbar
has the same content as bar
, and similarly for bazbaz
and foofoo
, then
rdfind -deleteduplicates true A B
results in
~/tmp% tree B
B
├── Bad
│ └── quux
├── Excellent
└── Good
Alternate solution in case your version of Ubuntu does not include rdfind:
You could instead use fdupes
:
sudo apt-get install fdupes
fdupes --recurse --delete --noprompt A B