How to clean duplicates existing in one folder from another recursively?

Below are two solutions, depending on how we define "duplicate":

  • Files with the same relative path, or
  • Files with the same content but not necessarily the same name

If by "duplicate" we mean two files which share the same relative path, then you could use find and xargs to remove the duplicates. For example, suppose you have

~/tmp% tree A
A
└── Excellent
    ├── bar
    ├── baz
    └── foo
~/tmp% tree B
B
├── Bad
│   └── quux
├── Excellent
│   ├── bar
│   ├── baz
│   └── foo
└── Good

Then

find /home/unutbu/tmp/A  -depth -type f -print0 | xargs -0 -I{} bash -c 'rm "/home/unutbu/tmp/B${1#*A}"' - {}

results in

~/tmp% tree B
B
├── Bad
│   └── quux
├── Excellent
└── Good

Or, if by "duplicate" we mean two files share the same content, though perhaps not the same filename, then you could use rdfind:

sudo apt-get install rdfind

If we have this directory structure:

~/tmp% tree A
A
└── Excellent
    ├── bar
    ├── baz
    └── foo

1 directory, 3 files
~/tmp% tree B
B
├── Bad
│   └── quux
├── Excellent
│   ├── barbar
│   ├── bazbaz
│   └── foofoo
└── Good

where barbar has the same content as bar, and similarly for bazbaz and foofoo, then

rdfind -deleteduplicates true A B

results in

~/tmp% tree B
B
├── Bad
│   └── quux
├── Excellent
└── Good

Alternate solution in case your version of Ubuntu does not include rdfind:

You could instead use fdupes:

sudo apt-get install fdupes
fdupes --recurse --delete --noprompt A B