How to make pip "dry-run"?

For developing a script that runs pip install it would be usefull to have a --dry-run function.

I came across the --no-install option. But this one is deprecated and on call references this. There are hints to unpack a package only, but I can't find a unpack option in the pip documentation.

Solution 1:

Yes - pip should have a dry-run option, to indicate what would happen in a complex situation. It is dangerous when running pip install downgrades packages without asking you. We need some way to ask what would happen if we run pip install -r requirements.txt without laboriously searching thru all the requirements and comparing them to the currently installed ones.

It looks like setup.py used to have a dry-run. Folks are asking for it elsewhere.

Some progress in that direction can be found here:

• Dry Run for Python Pip | BackSlasher
• nvie/pip-tools: A set of tools to keep your pinned Python dependencies fresh.

Update: Activity continues on the official bug related to this: Add `pip install --dry-run` or similar, to get resolution result · Issue #53 · pypa/pip. It is blocked on the development of the dependency resolver, which is nearing completion

Solution 2:

It appears you are right, it has been deprecated (ref).

If by trial run you mean testing it out before actually installing a package in a certain place, presumably before a system wide install, then you can simply run it sandboxed using a virtual environment and then simply discard the environment.

virtualenv /tmp/venv; /tmp/venv/bin/pip install flask; rm -rf /tmp/venv 

Not as succinct as using a dry-run argument to pip, but it does the job. Also if you want to do a dry run of a series of package installations, omit the deletion at the end.

In a script you can distil it into a procedure:

#!/bin/bash

TMP_DIR='/tmp/venv'

function dry_run (){
    if [ ! -d "$TMP_DIR" ]; then
            virtualenv /tmp/venv
    fi
    /tmp/venv/bin/pip install $1
}

dry_run flask
dry_run uwsgi
rm -rf $TMP_DIR

If you want to do a dry run that tests that the new install(s) play well with system wide deployed, then use virtualenv's system-site-packages option.

virtualenv --system-site-packages /tmp/venv; /tmp/venv/bin/pip install flask; rm -rf /tmp/venv

Solution 3:

The pip-sync command from pip-tools can report what packages would be installed, but will also output the ones that are installed but not in the requirements file. Its dry run option is -n

$ pip install pip-tools
$ pip-sync -n requirements.txt 
Would uninstall:
  pip-tools
Would install:
  requests

Here's the help from pip-sync:

pip-sync --help
Usage: pip-sync [OPTIONS] [SRC_FILES]...

  Synchronize virtual environment with requirements.txt.

Options:
  --version               Show the version and exit.
  -n, --dry-run           Only show what would happen, don't change anything
  --force                 Proceed even if conflicts are found
  -f, --find-links TEXT   Look for archives in this directory or on this HTML
                          page
  -i, --index-url TEXT    Change index URL (defaults to PyPI)
  --extra-index-url TEXT  Add additional index URL to search
  --trusted-host TEXT     Mark this host as trusted, even though it does not
                          have valid or any HTTPS.
  --no-index              Ignore package index (only looking at --find-links
                          URLs instead)
  -q, --quiet             Give less output
  --user                  Restrict attention to user directory
  --cert TEXT             Path to alternate CA bundle.
  --client-cert TEXT      Path to SSL client certificate, a single file
                          containing the private key and the certificate in
                          PEM format.
  --help                  Show this message and exit.

Solution 4:

This approach uses the current environment you are using and overrides paths pythons sees, it uses a similar approach to pipenv and virtualenv, but without any additional env creation, no changes to the current env, self-cleaning, and taking your current installed packages in the final dependency list solving algorythm, you know, just like a proper dry run.

1. pip has a --target flag that lets you point to some arbitrary directory to install packages to, instead of system packages
2. add that path to your PYTHONPATH temporarily, so you can figure out what was installed
3. if it fails, well, then it failed, otherwise you can just generate a freeze list
4. finally, a sweet diff for your cognitive convenience

So, the script would be something like this:

export DRY_RUN=/tmp/python-dry-run #points to the temp dir
mkdir -p $DRY_RUN #creates the dir
pip install -r requirements.txt --target $DRY_RUN #dry run happens here
PYTHONPATH=$DRY_RUN pip freeze > requirements-dry-run.txt #dry run output
diff --color requirements.txt requirements-dry-run.txt #red removed, green added
rm -rf $DRY_RUN #cleanup

caveats, warnings and opinions:

1. try not to export PYTHONPATH directly in your shell, that might make things confusing, instead pass it as a temp variable at the beginning of the command, just like I did there

2. pip freeze sucks, it usually leads to dependencies getting deadlocked in the future, so try your best to maintain your requirements file manually

3. if you're asking for this, you're probably in a dependency hell. An easy way of getting rid of requirements.txt deadlock nightmare is to just to get rid of mid and minor versions completely and letting pip figure out the dependencies with a bit more leeway. So, if you are using example==1.4.5 in your requirements.txt, make it example>=1

Solution 5:

[Ugly hack disclaimer] on Linux you can try to install in the system location as a user who does not have permission to install in the /usr/ directory (pip3 install --system on Debian/Ubuntu/etc.) The command fails with "Permission denied" but only after logging what is missing and what is not.

Makes you wonder how hard an actual dry-run option would really be to implement?