Combining conda environment.yml with pip requirements.txt
Solution 1:
Pip dependencies can be included in the environment.yml
file like this (docs):
# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- numpy=1.13.3 # pin version for conda
- pip:
# works for regular pip packages
- docx
- gooey
- matplotlib==2.0.0 # pin version for pip
# and for wheels
- http://www.lfd.uci.edu/~gohlke/pythonlibs/bofhrmxk/opencv_python-3.1.0-cp35-none-win_amd64.whl
It also works for .whl
files in the same directory (see Dengar's answer) as well as with common pip packages.
Solution 2:
One can also use the requirements.txt
directly in the YAML. For example,
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
- -r requirements.txt
Basically, any option you can run with pip install
you can run in a YAML. See the Advanced Pip Example for a showcase of other capabilities.
Important Note
A previous version of this answer (and Conda's Advanced Pip Example) used a substandard file
URI syntax:
- -r file:requirements.txt
Pip v21.2.1 introduced stricter behavior for URI parsing and no longer supports this. See this answer for details.
Solution 3:
Just want to add that adding a wheel in the directory also works. I was getting this error when using the entire URL:
HTTP error 404 while getting http://www.lfd.uci.edu/~gohlke/pythonlibs/f9r7rmd8/opencv_python-3.1.0-cp35-none-win_amd64.whl
Ended up downloading the wheel and saving it into the same directory as the yml file.
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
- opencv_python-3.1.0-cp35-none-win_amd64.whl
Solution 4:
If you want to do it automatically it seems that if you do:
conda env export > environment.yml`
already has the pip things you need. No need to run pip freeze > requirements4pip.txt
separately for me or include it as an
- pip:
- -r file:requirements.txt
as another answer mentioned.
See my yml file:
$ cat environment.yml
name: myenv
channels:
- pytorch
- dglteam
- defaults
- conda-forge
dependencies:
- _libgcc_mutex=0.1=main
- absl-py=0.12.0=py38h06a4308_0
- aiohttp=3.7.4=py38h27cfd23_1
- async-timeout=3.0.1=py38h06a4308_0
- attrs=20.3.0=pyhd3eb1b0_0
- beautifulsoup4=4.9.3=pyha847dfd_0
- blas=1.0=mkl
- blinker=1.4=py38h06a4308_0
- brotlipy=0.7.0=py38h27cfd23_1003
- bzip2=1.0.8=h7b6447c_0
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.4.13=h06a4308_1
- cachetools=4.2.1=pyhd3eb1b0_0
- cairo=1.14.12=h8948797_3
- certifi=2020.12.5=py38h06a4308_0
- cffi=1.14.0=py38h2e261b9_0
- chardet=3.0.4=py38h06a4308_1003
- click=7.1.2=pyhd3eb1b0_0
- conda=4.10.1=py38h06a4308_1
- conda-build=3.21.4=py38h06a4308_0
- conda-package-handling=1.7.3=py38h27cfd23_1
- coverage=5.5=py38h27cfd23_2
- cryptography=3.4.7=py38hd23ed53_0
- cudatoolkit=11.0.221=h6bb024c_0
- cycler=0.10.0=py38_0
- cython=0.29.23=py38h2531618_0
- dbus=1.13.18=hb2f20db_0
- decorator=4.4.2=pyhd3eb1b0_0
- dgl-cuda11.0=0.6.1=py38_0
- dill=0.3.3=pyhd3eb1b0_0
- expat=2.3.0=h2531618_2
- filelock=3.0.12=pyhd3eb1b0_1
- fontconfig=2.13.1=h6c09931_0
- freetype=2.10.4=h7ca028e_0
- fribidi=1.0.10=h7b6447c_0
- gettext=0.21.0=hf68c758_0
- glib=2.66.3=h58526e2_0
- glob2=0.7=pyhd3eb1b0_0
- google-auth=1.29.0=pyhd3eb1b0_0
- google-auth-oauthlib=0.4.4=pyhd3eb1b0_0
- graphite2=1.3.14=h23475e2_0
- graphviz=2.40.1=h21bd128_2
- grpcio=1.36.1=py38h2157cd5_1
- gst-plugins-base=1.14.0=h8213a91_2
- gstreamer=1.14.0=h28cd5cc_2
- harfbuzz=1.8.8=hffaf4a1_0
- icu=58.2=he6710b0_3
- idna=2.10=pyhd3eb1b0_0
- importlib-metadata=3.10.0=py38h06a4308_0
- intel-openmp=2021.2.0=h06a4308_610
- jinja2=2.11.3=pyhd3eb1b0_0
- joblib=1.0.1=pyhd3eb1b0_0
- jpeg=9b=h024ee3a_2
- kiwisolver=1.3.1=py38h2531618_0
- lcms2=2.12=h3be6417_0
- ld_impl_linux-64=2.33.1=h53a641e_7
- libarchive=3.4.2=h62408e4_0
- libffi=3.2.1=hf484d3e_1007
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libglib=2.66.3=hbe7bbb4_0
- libiconv=1.16=h516909a_0
- liblief=0.10.1=he6710b0_0
- libpng=1.6.37=h21135ba_2
- libprotobuf=3.14.0=h8c45485_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_1
- libuuid=1.0.3=h1bed415_2
- libuv=1.40.0=h7b6447c_0
- libxcb=1.14=h7b6447c_0
- libxml2=2.9.10=hb55368b_3
- lz4-c=1.9.2=he1b5a44_3
- markdown=3.3.4=py38h06a4308_0
- markupsafe=1.1.1=py38h7b6447c_0
- matplotlib=3.3.4=py38h06a4308_0
- matplotlib-base=3.3.4=py38h62a2d02_0
- mkl=2020.2=256
- mkl-service=2.3.0=py38h1e0a361_2
- mkl_fft=1.3.0=py38h54f3939_0
- mkl_random=1.2.0=py38hc5bc63f_1
- multidict=5.1.0=py38h27cfd23_2
- ncurses=6.2=he6710b0_1
- networkx=2.5.1=pyhd3eb1b0_0
- ninja=1.10.2=hff7bd54_1
- numpy=1.19.2=py38h54aff64_0
- numpy-base=1.19.2=py38hfa32c7d_0
- oauthlib=3.1.0=py_0
- olefile=0.46=pyh9f0ad1d_1
- openssl=1.1.1k=h27cfd23_0
- pandas=1.2.4=py38h2531618_0
- pango=1.42.4=h049681c_0
- patchelf=0.12=h2531618_1
- pcre=8.44=he6710b0_0
- pillow=8.2.0=py38he98fc37_0
- pip=21.0.1=py38h06a4308_0
- pixman=0.40.0=h7b6447c_0
- pkginfo=1.7.0=py38h06a4308_0
- protobuf=3.14.0=py38h2531618_1
- psutil=5.8.0=py38h27cfd23_1
- py-lief=0.10.1=py38h403a769_0
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.8=py_0
- pycosat=0.6.3=py38h7b6447c_1
- pycparser=2.20=py_2
- pyjwt=2.0.1=pyhd8ed1ab_1
- pyopenssl=20.0.1=pyhd3eb1b0_1
- pyparsing=2.4.7=pyhd3eb1b0_0
- pyqt=5.9.2=py38h05f1152_4
- pysocks=1.7.1=py38h06a4308_0
- python=3.8.2=hcf32534_0
- python-dateutil=2.8.1=pyhd3eb1b0_0
- python-libarchive-c=2.9=pyhd3eb1b0_1
- python_abi=3.8=1_cp38
- pytorch=1.7.1=py3.8_cuda11.0.221_cudnn8.0.5_0
- pytz=2021.1=pyhd3eb1b0_0
- pyyaml=5.4.1=py38h27cfd23_1
- qt=5.9.7=h5867ecd_1
- readline=8.1=h27cfd23_0
- requests=2.25.1=pyhd3eb1b0_0
- requests-oauthlib=1.3.0=py_0
- ripgrep=12.1.1=0
- rsa=4.7.2=pyhd3eb1b0_1
- ruamel_yaml=0.15.100=py38h27cfd23_0
- scikit-learn=0.24.1=py38ha9443f7_0
- scipy=1.6.2=py38h91f5cce_0
- setuptools=52.0.0=py38h06a4308_0
- sip=4.19.13=py38he6710b0_0
- six=1.15.0=pyh9f0ad1d_0
- soupsieve=2.2.1=pyhd3eb1b0_0
- sqlite=3.35.4=hdfb4753_0
- tensorboard=2.4.0=pyhc547734_0
- tensorboard-plugin-wit=1.6.0=py_0
- threadpoolctl=2.1.0=pyh5ca1d4c_0
- tk=8.6.10=hbc83047_0
- torchaudio=0.7.2=py38
- torchtext=0.8.1=py38
- torchvision=0.8.2=py38_cu110
- tornado=6.1=py38h27cfd23_0
- typing-extensions=3.7.4.3=0
- typing_extensions=3.7.4.3=py_0
- urllib3=1.26.4=pyhd3eb1b0_0
- werkzeug=1.0.1=pyhd3eb1b0_0
- wheel=0.36.2=pyhd3eb1b0_0
- xz=5.2.5=h7b6447c_0
- yaml=0.2.5=h7b6447c_0
- yarl=1.6.3=py38h27cfd23_0
- zipp=3.4.1=pyhd3eb1b0_0
- zlib=1.2.11=h7b6447c_3
- zstd=1.4.5=h9ceee32_0
- pip:
- aioconsole==0.3.1
- lark-parser==0.6.5
- lmdb==0.94
- pexpect==4.6.0
- progressbar2==3.39.3
- ptyprocess==0.7.0
- pycapnp==1.0.0
- python-utils==2.5.6
- sexpdata==0.0.3
- tqdm==4.56.0
prefix: /home/miranda9/miniconda3/envs/myenv
note that at the time of this writing doing conda env create --file environment.yml
to create the yml env results in an error:
$ conda env create --file environment.yml
CondaValueError: prefix already exists: /home/miranda9/miniconda3/envs/myenv