What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?
Solution 1:
Recommendation for beginners:
This is my personal recommendation for beginners: start by learning virtualenv
and pip
, tools which work with both Python 2 and 3 and in a variety of situations, and pick up other tools once you start needing them.
PyPI packages not in the standard library:
-
virtualenv
is a very popular tool that creates isolated Python environments for Python libraries. If you're not familiar with this tool, I highly recommend learning it, as it is a very useful tool, and I'll be making comparisons to it for the rest of this answer.
It works by installing a bunch of files in a directory (eg: env/
), and then modifying the PATH
environment variable to prefix it with a custom bin
directory (eg: env/bin/
). An exact copy of the python
or python3
binary is placed in this directory, but Python is programmed to look for libraries relative to its path first, in the environment directory. It's not part of Python's standard library, but is officially blessed by the PyPA (Python Packaging Authority). Once activated, you can install packages in the virtual environment using pip
.
-
pyenv
is used to isolate Python versions. For example, you may want to test your code against Python 2.7, 3.6, 3.7 and 3.8, so you'll need a way to switch between them. Once activated, it prefixes thePATH
environment variable with~/.pyenv/shims
, where there are special files matching the Python commands (python
,pip
). These are not copies of the Python-shipped commands; they are special scripts that decide on the fly which version of Python to run based on thePYENV_VERSION
environment variable, or the.python-version
file, or the~/.pyenv/version
file.pyenv
also makes the process of downloading and installing multiple Python versions easier, using the commandpyenv install
. -
pyenv-virtualenv
is a plugin forpyenv
by the same author aspyenv
, to allow you to usepyenv
andvirtualenv
at the same time conveniently. However, if you're using Python 3.3 or later,pyenv-virtualenv
will try to runpython -m venv
if it is available, instead ofvirtualenv
. You can usevirtualenv
andpyenv
together withoutpyenv-virtualenv
, if you don't want the convenience features. -
virtualenvwrapper
is a set of extensions tovirtualenv
(see docs). It gives you commands likemkvirtualenv
,lssitepackages
, and especiallyworkon
for switching between differentvirtualenv
directories. This tool is especially useful if you want multiplevirtualenv
directories. -
pyenv-virtualenvwrapper
is a plugin forpyenv
by the same author aspyenv
, to conveniently integratevirtualenvwrapper
intopyenv
. -
pipenv
aims to combinePipfile
,pip
andvirtualenv
into one command on the command-line. Thevirtualenv
directory typically gets placed in~/.local/share/virtualenvs/XXX
, withXXX
being a hash of the path of the project directory. This is different fromvirtualenv
, where the directory is typically in the current working directory.pipenv
is meant to be used when developing Python applications (as opposed to libraries). There are alternatives topipenv
, such aspoetry
, which I won't list here since this question is only about the packages that are similarly named.
Standard library:
-
pyvenv
(not to be confused withpyenv
in the previous section) is a script shipped with Python 3 but deprecated in Python 3.6 as it had problems (not to mention the confusing name). In Python 3.6+, the exact equivalent ispython3 -m venv
. -
venv
is a package shipped with Python 3, which you can run usingpython3 -m venv
(although for some reason some distros separate it out into a separate distro package, such aspython3-venv
on Ubuntu/Debian). It serves the same purpose asvirtualenv
, but only has a subset of its features (see a comparison here).virtualenv
continues to be more popular thanvenv
, especially since the former supports both Python 2 and 3.
Solution 2:
I would just avoid the use of virtualenv
after Python3.3+ and instead use the standard shipped library venv
. To create a new virtual environment you would type:
$ python3 -m venv <MYVENV>
virtualenv
tries to copy the Python binary into the virtual environment's bin directory. However it does not update library file links embedded into that binary, so if you build Python from source into a non-system directory with relative path names, the Python binary breaks. Since this is how you make a copy distributable Python, it is a big flaw. BTW to inspect embedded library file links on OS X, use otool
. For example from within your virtual environment, type:
$ otool -L bin/python
python:
@executable_path/../Python (compatibility version 3.4.0, current version 3.4.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
Consequently I would avoid virtualenvwrapper
and pipenv
. pyvenv
is deprecated. pyenv
seems to be used often where virtualenv
is used but I would stay away from it also since I think venv
also does what pyenv
is built for.
venv
creates virtual environments in the shell that are fresh and sandboxed, with user-installable libraries, and it's multi-python safe.
Fresh: because virtual environments only start with the standard libraries that ship with python, you have to install any other libraries all over again with pip install
while the virtual environment is active.
Sandboxed: because none of these new library installs are visible outside the virtual environment, so you can delete the whole environment and start again without worrying about impacting your base python install.
User-installable libraries: because the virtual environment's target folder is created without sudo
in some directory you already own, so you won't need sudo
permissions to install libraries into it.
multi-python safe: because when virtual environments activate, the shell only sees the python version (3.4, 3.5 etc.) that was used to build that virtual environment.
pyenv
is similar to venv
in that it lets you manage multiple python environments. However with pyenv
you can't conveniently rollback library installs to some start state and you will likely need admin
privileges at some point to update libraries. So I think it is also best to use venv
.
In the last couple of years I have found many problems in build systems (emacs packages, python standalone application builders, installers...) that ultimately come down to issues with virtualenv
. I think python will be a better platform when we eliminate this additional option and only use venv
.
EDIT: Tweet of the BDFL,
I use venv (in the stdlib) and a bunch of shell aliases to quickly switch.
— Guido van Rossum (@gvanrossum) October 22, 2020
Solution 3:
UPDATE 20200825:
Added below "Conclusion" paragraph
I've went down the pipenv
rabbit hole (it's a deep and dark hole indeed...) and since the last answer is over 2 years ago, felt it was useful to update the discussion with the latest developments on the Python virtual envelopes topic I've found.
DISCLAIMER:
This answer is NOT about continuing the raging debate about the merits of pipenv versus venv as envelope solutions- I make no endorsement of either. It's about PyPA endorsing conflicting standards and how future development of virtualenv promises to negate making an either/or choice between them at all. I focused on these two tools precisely because they are the anointed ones by PyPA.
venv
As the OP notes, venv is a tool for virtualizing environments. NOT a third party solution, but native tool. PyPA endorses venv for creating VIRTUAL ENVELOPES: "Changed in version 3.5: The use of venv is now recommended for creating virtual environments".
pipenv
pipenv- like venv - can be used to create virtual envelopes but additionally rolls-in package management and vulnerability checking functionality. Instead of using requirements.txt
, pipenv
delivers package management via Pipfile. As PyPA endorses pipenv for PACKAGE MANAGEMENT, that would seem to imply pipfile
is to supplant requirements.txt
.
HOWEVER: pipenv uses virtualenv as its tool for creating virtual envelopes, NOT venv which is endorsed by PyPA as the go-to tool for creating virtual envelopes.
Conflicting Standards:
So if settling on a virtual envelope solution wasn't difficult enough, we now have PyPA endorsing two different tools which use different virtual envelope solutions. The raging Github debate on venv vs virtualenv which highlights this conflict can be found here.
Conflict Resolution:
The Github debate referenced in above link has steered virtualenv development in the direction of accommodating venv in future releases:
prefer built-in venv: if the target python has venv we'll create the environment using that (and then perform subsequent operations on that to facilitate other guarantees we offer)
Conclusion:
So it looks like there will be some future convergence between the two rival virtual envelope solutions, but as of now pipenv- which uses virtualenv
- varies materially from venv
.
Given the problems pipenv solves and the fact that PyPA has given its blessing, it appears to have a bright future. And if virtualenv delivers on its proposed development objectives, choosing a virtual envelope solution should no longer be a case of either pipenv OR venv.
Update 20200825:
An oft repeated criticism of Pipenv I saw when producing this analysis was that it was not actively maintained. Indeed, what's the point of using a solution whose future could be seen questionable due to lack of continuous development? After a dry spell of about 18 months, Pipenv is once again being actively developed. Indeed, large and material updates have since been released.