Using Python with homebrew on OS X

I have been using Mac python for a while, and I decided to teach myself matplotlib, because I want to have the experience with some common modules. I hear from everyone that once you get into non-standard modules, it's best to use python threw homebrew, so you have access to pip and not easy_install. After running: $brew install python --with-brewed-openssl, $brew install python3 --with-brewed-openssl, and $pip install matplotlib, I go to the python shell with $python3. Once there, I run import matplotlib.pyplot as plt, and get the following:

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 00:54:21) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib.pyplot as plt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'matplotlib'
>>>

I have a feeling I'm still using the mac python, and not the brewed python, and I have tried editing the path, but it isn't working, even though I am running $source ~/.bash_profile after every edit. Just in case I am editing the path wrong, I will post the file contents below:

PATH=/usr/local/bin:/usr/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin
PATH=/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin
export MSF_DATABASE_CONFIG=/usr/local/share/metasploit-framework/database.yml

PATH="/Users/ericmarkmartin/.apportable/SDK/bin:$PATH"

PATH="/Users/ericmarkmartin/.apportable/SDK/bin:$PATH"

# Setting PATH for Python 3.4
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.4/bin:${PATH}"
PATH=“/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/bin:${PATH}”
export PATH
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/Users/ericmarkmartin/.rvm/bin
PATH=/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/Users/ericmarkmartin/.rvm/bin
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/Users/ericmarkmartin/.rvm/bin
PATH=/usr/local/bin:/usr/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/local/sbin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/Users/ericmarkmartin/.rvm/bin:/Users/ericmarkmartin/.rvm/bin
PATH=/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/local/sbin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/Users/ericmarkmartin/.apportable/SDK/bin:/usr/local/opt/ruby193/bin:/usr/local/bin:/usr/local/sbin:/opt/iOSOpenDev/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/Users/ericmarkmartin/.rvm/bin:/Users/ericmarkmartin/.rvm/bin

[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function*
[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm" # Load RVM function

# Setting PATH for MacPython 2.6
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.6/bin:${PATH}"
export PATH
export PATH=/usr/local/bin:/usr/local/sbin:${PATH}

$pip3 install matplotlib returns the following

RuntimeError: Broken toolchain: cannot link a simple C program

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /private/var/folders/sr/j0s763cj06v38c6btg6v7k7r0000gn/T/pip_build_ericmarkmartin/matplotlib
Storing debug log for failure in /Users/ericmarkmartin/.pip/pip.log

If more information is needed, please feel free to ask, and thank you so much in advance for the help!