Do python projects need a MANIFEST.in, and what should be in it?

The "Python Distribute" guide (was at python-distribute.org, but that registration has lapsed) tells me to include doc/txt files and .py files are excluded in MANIFEST.in file

The sourcedist documentation tells me only sdist uses MANIFEST.in and only includes file you specify and to include .py files. It also tells me to use: python setup.py sdist --manifest-only to generate a MANIFEST, but python tells me this doesn't exist

I appreciate these are from different versions of python and the distribution system is in a complete mess, but assuming I am using python 3 and setuptools (the new one that includes distribute but now called setuptools, not the old setuptools that was deprecated for distribute tools only to be brought back into distribute and distribute renamed to setuptools.....)

and I'm following the 'standard' folder structure and setup.py file,

Do I need a MANIFEST.in ?
What should be in it ?
When will all these different package systems and methods be made into one single simple process ?

Solution 1:

Re: "Do I need a MANIFEST.in?

No, you do not have to use MANIFEST.in. Both, distutils and setuptools are including in source distribution package all the files mentioned in setup.py - modules, package python files, README.txt and test/test*.py. If this is all you want to have in distribution package, you do not have to use MANIFEST.in.

If you want to manipulate (add or remove) default files to include, you have to use MANIFEST.in.

Re: What should be in it?

The procedure is simple:

Make sure, in your setup.py you include (by means of setup arguments) all the files you feel important for the program to run (modules, packages, scripts ...)
Clarify, if there are some files to add or some files to exclude. If neither is needed, then there is no need for using MANIFEST.in.
If MANIFEST.in is needed, create it. Usually, you add there tests*/*.py files, README.rst if you do not use README.txt, docs files and possibly some data files for test suite, if necessary.

For example:

include README.rst
include COPYING.txt

To test it, run python setup.py sdist, and examine the tarball created under dist/.

When will all these different package systems ...

Comparing the situation today and 2 years ago - the situation is much much better - setuptools is the way to go. You can ignore the fact, distutils is a bit broken and is low level base for setuptools as setuptools shall take care of hiding these things from you.

EDIT: Last few projects I use pbr for building distribution packages with three line setup.py and rest being in setup.cfg and requirements.txt. No need to care about MANIFEST.in and other strange stuff. Even though the package would deserve a bit more documentation. See http://docs.openstack.org/developer/pbr/

Solution 2:

Old question, new answer:

No, you don't need MANIFEST.in. However, to get setuptools to do what you (usually) mean, you do need to use the setuptools_scm, which takes the role of MANIFEST.in in 2 key places:

It ensures all relevant files are packaged when running the sdist command (where all relevant files is defined as "all files under source control")
When using include_package_data to include package data as part of the build or bdist_wheel. (again: files under source control)

The historical understanding of MANIFEST.in is: when you don't have a source control system, you need some other mechanism to distinguish between "source files" and "files that happen to be in your working directory". However, your project is under source control (right??) so there's no need for MANIFEST.in. More info in this article.