Making an executable in Cython

Been playing with cython. Normally program in Python, but used C in a previous life. I can't figure out how to make a free-standing executable.

I've downloaded cython, and I can make a .pyx file (that's just a normal Python file with a .pyx extension), that executes in the Python shell, using: import pyximport; pyximport.install()

I can generate a .c file at the command line with: cython file.pyx I can generate a .so file by building a standard setup.py and executing:

setup.py build_ext --inplace

I've tried making an executable out of the .so file using gcc with various options, but always have tons of missing files, headers, etc. Have tried pointing to headers from virtually everywhere, but with no success, and am not really familiar with what all the gcc options do, or even if I should be using gcc.

I've having a disconnect here with the fact that I can run my program in the Python shell, but not at the command line, (I don't want users to have to get into the shell, import modules, etc).

What am I missing here?

Solution 1:

What you want is the --embed flag for the Cython compiler. There isn't a ton of documentation on it, but this is what I was able to find. It does link to a simple working example.

To compile the Cython source code to a C file that can then be compiled to an executable you use a command like cython myfile.pyx --embed and then compile with whichever C compiler you are using.

When you compile the C source code, you will still need to include the directory with the Python headers and link to the corresponding Python shared library on your system (a file named something like libpython27.so or libpython27.a if you are using Python 2.7).

Edit: Here are some more instructions on how to get the commands for including the proper headers and linking against the proper libraries.

As I said earlier, you need to run the Cython compiler like this:

cython <cython_file> --embed

To compile using gcc, you will need to find where the python headers are on your system (you can get this location by running distutils.sysconfig.get_python_inc() (you'll have to import it first). It is probably just the /include subdirectory in your Python installation directory.

You will also have to find the python shared library. For Python 2.7 it would be libpython27.a on Windows or libpython2.7.so on Linux.

Your gcc command will then be

gcc <C_file_from_cython> -I<include_directory> -L<directory_containing_libpython> -l<name_of_libpython_without_lib_on_the_front> -o <output_file_name>

It may be wise to include the -fPIC flag. On Windows 64 bit machines you will also have to include the flags -D MS_WIN64 that tells mingw to compile for 64 bit windows.

If you are compiling something that depends on NumPy, you will also need to include the directory containing the NumPy headers. You can find this folder by running numpy.get_include() (again, after importing numpy). Your gcc command then becomes

gcc <C_file_from_cython> -I<include_directory> -I<numpy_include_directory> -L<directory_containing_libpython> -l<name_of_libpython_without_lib_on_the_front> -o <output_file_name>

This gcc command option guide may be helpful.

Also, I would recommend you use Cython memory views if possible. That will make it so that you won't have to include the NumPy headers and include the NumPy pxd file in your Cython file. It also makes slicing operations easier for the C compiler to optimize.

Solution 2:

Tested this on Ubuntu:

Install Cython using the following (Python 2):

sudo apt-get install cython

For Python 3:

sudo apt-get install cython3

To compile the Python code into a C code run (change cython to cython3 for Python 3):

cython --embed -o example.c example.py

This will generate the example.c file.

Now compile the example.c file:

gcc -Os -I /usr/include/python2.7 example.c -lpython2.7 -o example

Run the file:

./example

Now for Python 3, something like this will work (not tested):

gcc -Os -I /usr/include/python3.6 example.c -lpython3.6 -o example

where python3.x is the version of Python installed on your machine.

Solution 3:

This is a solution for Windows + MS Visual Studio 14 (as no-one mentioned the cl.exe parameters yet).

First use the embed parameter to produce the test.c file:

cython test.pyx --embed

Then compile it:

call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x64
cl test.c /I C:\Python37\include /link C:\Python37\libs\python37.lib

The output is small executable file test.exe (140 KB in my case for a print("Hello World")).

Notes:

• for a GUI app (and no console app), see @longgangfan's answer Can Cython compile to an EXE?, i.e. add /subsystem:windows /entry:wmainCRTStartup in the cl.exe parameters.

• for more information about whether the source code is still there, see Are executables produced with Cython really free of the source code? and Does compiled standalone Cython executable still contain all original source code?.

• also, the .exe won't run out-of-the-box on any machine; some other files are required, see Minimal set of files required to distribute an embed-Cython-compiled code and make it work on any machine.