How to build and install Python 3.7.x from source on Debian 9.8

Solution 1:

Install git

$ sudo apt install git

Install Python build dependencies

Python includes a set of modules that it builds by linking to other popular open source projects. Many of those projects are included in Debian by default as executable programs, but the development files (headers, libraries) necessary to link them into Python are packaged separately and not included in a default Debian install.

Some of these libraries are actually required for the python build/install steps to complete successfully:

| Library                            | Python Module | Dev Package  |
| ---------------------------------- | ------------- | ------------ |
| https://www.zlib.net/              | `zlib`        | `zlib1g-dev` |
| https://www.sourceware.org/libffi/ | `_ctypes`     | `libffi-dev` |

The entries in the 'Dev Package' column are the names of packages containing the development files our python build needs.

Each of these 'dev' packages has a corresponding binary package that is probably already installed. So you're likely not going to be adding new pieces of software to your system by installing these. You're just installing the files necessary to compile new pieces of software (in our case a newer python) so that it can use them. (Also note that these 'dev' packages all have their corresponding binary packages as dependencies, so installing a dev package will ensure that its binary package is also installed.)

So let's install them:

sudo apt install zlib1g-dev libffi-dev

Next we have the OpenSSL library. Python considers OpenSSL optional, but you probably want it. For example, even using Python's package installation tools can run into trouble when fetching https urls if SSL/TLS support is missing.

| Library                  | Python Module | Dev Package |
| ------------------------ | ------------- | ----------- |
| https://www.openssl.org/ | `_ssl`        | `libssl-dev |

Let's install it:

sudo apt install libssl-dev

The next set of packages fall more in the 'optional' category:

| Library                                              | Python Module(s)              | Dev Package        |
| ---------------------------------------------------- | ----------------------------- | ------------------ |
| http://www.bzip.org/                                 | `_bz2`                        | `libbz2-dev`       |
| https://www.gnu.org/software/ncurses/                | `_curses` and `_curses_panel` | `libncursesw5-dev` |
| https://www.gnu.org.ua/software/gdbm/                | `_dbm` and `_gdbm`            | `libgdbm-dev`      |
| https://tukaani.org/xz/                              | `_lzma`                       | `liblzma-dev`      |
| https://www.sqlite.org/                              | `_sqlite3`                    | `libsqlite3-dev`   |
| https://www.tcl.tk/software/tcltk/                   | `_tkinter`                    | `tk-dev`           |
| https://github.com/karelzak/util-linux               | `_uuid`                       | `uuid-dev`         |
| https://tiswww.case.edu/php/chet/readline/rltop.html | `readline`                    | `libreadline-dev`  |

Python can build and install without these, and your applications might not need them. On the other hand, if you install them, the 'make' step shouldn't report any modules it was unable to build, and the binaries are already included in a default Debian install.

The command to copy-paste if you'd like to go ahead and install all of the above 'optional' packages is:

sudo apt install libbz2-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev tk-dev uuid-dev libreadline-dev

Build and install python from source

1. clone the python git repository.

$ git clone [email protected]:python/cpython.git
Cloning into 'cpython'...
[...]
Resolving deltas: 100% (592215/592215), done.

2. Checkout the version you want to install (in this case, 3.7.2) using the git tag name.

$ cd cpython/
cpython$ git checkout v3.7.2
Note: checking out 'v3.7.2'.
[...]
HEAD is now at 9a3ffc0492... 3.7.2final
cpython$

3. Choose an install prefix. That is, the path the compiled and linked project will be installed into. In this case I'll use $HOME/python/v3.7.2. I'm including the version number since I'll want to install other versions in the future, and keep them separate. Run the configure script with this path as the --prefix argument. (If you don't provide a --prefix argument, it will default to /usr/local).

cpython$ ./configure --prefix=$HOME/python/v3.7.2
[...]
checking for inflateCopy in -lz... yes
[...]
checking for openssl/ssl.h in /usr... yes
[...]

The configure script will check a bunch of things, many of which are unnecessary. The ones shown in the above output, though, can be considered necessary.

4. Build python by running make. If we included the minimum set of dependencies necessary to build and install python, the output will tell you near the end what modules were not built:

cpython$ make
[...]
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2                  _curses               _curses_panel      
_dbm                  _gdbm                 _lzma              
_sqlite3              _tkinter              _uuid              
readline                                                       
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

The make output describes this as a list of 'optional' modules but again, at least for this particular version of python, it probably won't build and install successfully without the zlib, and _ctypes modules.

5. Install python by running make install. We've already provided the install location back in the configure step. If your install location is a root-owned directory (e.g. /usr/local), prefix this command with sudo. Since in this case I'm installing to a user-owned directory, I don't want to do that.

cpython$ make install
Creating directory /home/python/v3.7.2/bin
Creating directory /home/python/v3.7.2/lib
[...]

6. You should now have an executable you can run from the bin directory underneath the install prefix named python3. (Note that this installer doesn't put anything at bin/python; just bin/python3.)

$ $HOME/python/v3.7.2/bin/python3
Python 3.7.2 (v3.7.2:9a3ffc0492, Mar 10 2019, 19:35:56) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
$