Python 3.9 does not find packages that work with Python 3.8

I have a hard time understanding how Python is installed on my Mac. Not sure if it is important, but I have a Macbook Pro with an M1 Pro chip and come from years of using Linux with a package-manager and thus mostly use iTerm 2 together with brew to manage my installations.

When I open up iTerm and enter the command python3 I am greeted with

❯ python3
Python 3.8.9 (default, Oct 26 2021, 07:25:53)
[Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> 

I can import packages that I have installed with pip3. But recently I installed PyQt5 over the command brew install pyqt5 and as a dependency, it installed Python 3.9.9 alongside it. Now I can also use the command python 3.9 and I get

❯ python3.9
Python 3.9.9 (main, Nov 21 2021, 03:16:13)
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>>

I can see that 3.8.9 is the default, which I find a bit weird (I would have expected that the newest stable release version gets used as default), but I don't understand at all why the packages I installed with pip3 can be imported by python3.8.9 but not by python3.9.9... I'm hesitant to remove any versions of python because I know that macOS depends on some of them... Could somebody shed some light on what is going on here and how I can set the newest version as the default version and get it to recognize the installed packages?

Solution 1:

You have two separate installations of python3 and they know nothing of each other,

One from Apple the original in /usr/bin/python3 the 3.8.9 version

The other via Homebrew (I suspect in /opt/homebrew/python3) the 3.9.9

They know nothing of each other.

pip3 is a script file that is related to the python version it works with - the first line of pip3 is #!<path>/python3

To use the newer homebrew version you need pip3 from Homebrew and then uses that pip3 to install the python packages you installed for the Apple version.

The version of python (and pip) you end up using is the version that is first on your $PATH - see which python3 so see what the shell thinks

If you really want to use python you should also look at the official python way of using multiple python versions and package versions - venv https://docs.python.org/3/library/venv.html