Python Interpreter only using 12% CPU Power

I'm using python on ubuntu for text analysis. Despite the severe amount of work the program is doing the CPU usage as shown in the system monitor stays consistently at 12%.

I changed the priority of the program from Normal to Very High but that had no effect.

What is limiting the amount of CPU usage which my python program can get and how can I change that, so the program can utilize more cpu power?

Solution 1:

I assume you have a CPU with 8 virtual cores (quad-core with hyper-threading probably)? That means one fully loaded CPU thread/virtual core equals 12.5% total load.

The Python interpreter is an application which only runs as one single process by default and is therefore not able to take advantage of more than one virtual core. Even if the code you run with it uses multithreading, it will still only use one CPU thread/virtual core, because of the GIL (global interpreter lock).

Only if your Python program uses multiprocessing, which in fact starts up multiple instances of the Python interpreter and lets them perform your tasks truly parallel, you can take advantage of multiple virtual cores/CPU threads. (As @SargeBorsch pointed out in his comment, there are also some advanced ways to achieve this without multiprocessing, but that's normally not something you quickly write yourself.)

Solution 2:

Another possibility, less likely in this case, is that the program is disk-bound, i.e. it is reading and writing to/from the disk which is slow, and the CPU is waiting for the disk.