$PATH is filled with duplicates

I recently started configuring a OSX 10.7 environment for development. I've installed HomeBrew, and added the following to my .bash_profile:

export PATH="/usr/local/bin:/usr/local/sbin:~/bin:$PATH"        

Everything is working great except when I echo $PATH I get the following string of duplicates: /usr/local/bin:/usr/local/sbin:~/bin:/usr/local/bin:/usr/local/sbin:~/bin:/usr/local/git/bin:usr/local/bin:/usr/local/git/bin:usr/local/bin:/usr/local/git/bin:usr/local/bin:/usr/local/git/bin:usr/local/bin:/usr/local/git/bin:usr/local/bin:/usr/local/git/bin:usr/local/bin:/usr/local/git/bin:usr/local/bin:/usr/local/git/bin:/usr/local/git/bin:/usr/local/git/bin:/usr/local/git/bin:/usr/local/git/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

To make it more readable, these are all the paths:

/usr/local/bin
/usr/local/sbin
~/bin:/usr/local/bin
/usr/local/sbin
~/bin
/usr/local/git/bin
usr/local/bin
/usr/local/git/bin
usr/local/bin
/usr/local/git/bin
usr/local/bin
/usr/local/git/bin
usr/local/bin
/usr/local/git/bin
usr/local/bin
/usr/local/git/bin
usr/local/bin
/usr/local/git/bin
usr/local/bin
/usr/local/git/bin
/usr/local/git/bin
/usr/local/git/bin
/usr/local/git/bin
/usr/local/git/bin
/usr/bin
/bin
/usr/sbin
/sbin
/usr/local/bin
/usr/X11/bin

I don't think I've made any changes to path besides the single line in my .bash_profile. What is the best way for me to pair down those duplicates? Is there a way to hunt down which files are modifying my path and try to eliminate them?


Having made my comment, here are some suggestions for pruning $PATH anyway.

Looking at your path, it appears that .bash_profile is being executed twice, or the modification made in .bash_profile is duplicated elsewhere.

There are 7 duplicate additions of '/usr/local/git/bin' and 'use/local/bin' [sic], followed by 5 more copies of '/usr/local/git/bin'. Depending on how git is installed, you may be able to query your package manager about what files were installed with git; there could be some configuration files or modified system files that affect PATH.

Put set -x at the very top of your .bash_login, then start a new login shell. You should get a lot of output that shows exactly what bash is doing on startup, which should help you figure out where PATH is being modified. You can remove set -x once you figure it out or give up. If you don't find anything, you could also add it to the beginning of /etc/profile to trace what the system does before your own .bash_profile is processed.


I ran Ryan Thompson's script from the command line.
It removed all the duplicates for me, without changing the order, and without leaving a trailing :

PATH="$(perl -e 'print join(":", grep { not $seen{$_}++ } split(/:/, $ENV{PATH}))')"

In addition to the convenient 1-liner above, Ryan shares the (more structured) script he uses in his config to de-duplicate other variables and the PATH.

(See his Unix & Linux post for more details)


I added this to my $HOME/.bashrc (you should also be able to add it to .bash_profile instead if you wish) to remove duplicate entries from the $PATH. I've only tested on linux but should also work on mac. It should be added after the initial export PATH.

export PATH=$(echo $PATH | awk -F: '
{ for (i = 1; i <= NF; i++) arr[$i]; }
END { for (i in arr) printf "%s:" , i; printf "\n"; } ')