What is __pycache__?

From what I understand, a cache is an encrypted file of similar files.

What do we do with the __pycache__ folder? Is it what we give to people instead of our source code? Is it just my input data? This folder keeps getting created, what it is for?

Solution 1:

When you run a program in Python, the interpreter compiles it to bytecode first (this is an oversimplification) and stores it in the __pycache__ folder. If you look in there you will find a bunch of files sharing the names of the .py files in your project's folder, only their extensions will be either .pyc or .pyo. These are bytecode-compiled and optimized bytecode-compiled versions of your program's files, respectively.

As a programmer, you can largely just ignore it... All it does is make your program start a little faster. When your scripts change, they will be recompiled, and if you delete the files or the whole folder and run your program again, they will reappear (unless you specifically suppress that behavior).

When you're sending your code to other people, the common practice is to delete that folder, but it doesn't really matter whether you do or don't. When you're using version control (git), this folder is typically listed in the ignore file (.gitignore) and thus not included.

If you are using CPython (which is the most common, as it's the reference implementation) and you don't want that folder, then you can suppress it by starting the interpreter with the -B flag, for example

python -B foo.py

Another option, as noted by tcaswell, is to set the environment variable PYTHONDONTWRITEBYTECODE to any value (according to Python's man page, any "non-empty string").

Solution 2:

__pycache__ is a folder containing Python 3 bytecode compiled and ready to be executed.

I don't recommend routinely laboriously deleting these files or suppressing creation during development as it wastes your time. Just have a recursive command ready (see below) to clean up when needed as bytecode can become stale in edge cases (see comments).

Python programmers usually ignore bytecode. Indeed __pycache__ and *.pyc are common lines to see in .gitignore files. Bytecode is not meant for distribution and can be disassembled using dis module.

If you are using OS X you can easily hide all of these folders in your project by running following command from the root folder of your project.

find . -name '__pycache__' -exec chflags hidden {} \;

Replace __pycache__ with *.pyc for Python 2.

This sets a flag on all those directories (.pyc files) telling Finder/Textmate 2 to exclude them from listings. Importantly the bytecode is there, it's just hidden.

Rerun the command if you create new modules and wish to hide new bytecode or if you delete the hidden bytecode files.

On Windows the equivalent command might be (not tested, batch script welcome):

dir * /s/b | findstr __pycache__ | attrib +h +s +r

Which is same as going through the project hiding folders using right-click > hide...

Running unit tests is one scenario (more in comments) where deleting the *.pyc files and __pycache__ folders is indeed useful. I use the following lines in my ~/.bash_profile and just run cl to clean up when needed.

alias cpy='find . -name "__pycache__" -delete'
alias cpc='find . -name "*.pyc"       -delete'
...
alias cl='cpy && cpc && ...'

and more lately

# pip install pyclean
pyclean .