How to specify channel-specific package dependencies in conda package meta.yaml? [duplicate]

I am writing a package for conda-forge and needed to specify a dependency on another conda-forge dependency. Essentially, I need to install a pinned version of the conda-forge gdal package, because it actually compiles the version of libtiff that supports BIGTIFF files....

Now if I was installing gdal into a conda environment, I would write something like.

conda install -c conda-forge gdal=2.4.4 

I would to get this version of gdal=2.4.4 from conda-forge installed when installing the package. Now in the meta.yaml file, I can specify package dependencies like so, but I did not see how to specify a URL to a tar file, or whatever would work.

{% set version = "0.0.1" %}

package:
  name: mypackage
  version: {{ version }}

source:
  url: https://github.com/myrepo/{{ version }}.tar.gz
  sha256: ****6a63

build:
  number: 1
  skip: true  # [win and py27]
  entry_points:
    - mycli = mypackage.main:main

requirements:
  build:
    - python
    - 
  host:
    - python
    - pip
    - numpy
    - gdal  # <----- want to specify from conda-forge
  run:
    - python
    - gdal  # <----- want to specify from conda-forge

Any suggestions about how to do this would be appreciated.

Solution 1:

I don't think it's possible to specify the channel in meta.yaml. The following issue is still unresolved in the conda-build issue tracker: https://github.com/conda/conda-build/issues/532

As a workaround, if you know the exact version of gdal that you need, you can specify the exact version and "build string" in the recipe.

The only annoying thing is that you'll have to list gdal once for every combination of platform and python version your recipe needs to support.

requirements:
  build:
    - python
    - 
  host:
    - python
    - pip
    - numpy
    - gdal  2.4.4 py36h02fde04_1  # [osx and py==36]
    - gdal  2.4.4 py37h622575a_1  # [osx and py==37]
    - gdal  2.4.4 py38h57202bd_1  # [osx and py==38]
    - gdal  2.4.4 py36hbb8311d_1  # [linux and py==36]
    - gdal  2.4.4 py37hf8c3989_1  # [linux and py==37]
    - gdal  2.4.4 py38hfe926b7_1  # [linux and py==38]

  run:
    - python
    - gdal  2.4.4 py36h02fde04_1  # [osx and py==36]
    - gdal  2.4.4 py37h622575a_1  # [osx and py==37]
    - gdal  2.4.4 py38h57202bd_1  # [osx and py==38]
    - gdal  2.4.4 py36hbb8311d_1  # [linux and py==36]
    - gdal  2.4.4 py37hf8c3989_1  # [linux and py==37]
    - gdal  2.4.4 py38hfe926b7_1  # [linux and py==38]

(I copied those from the gdal package listing on the conda-forge channel.)

BTW, since you mentioned that the real important difference for you is libtiff, then should you be pinning libtiff instead of gdal? Or maybe both?

Edit:

It would be nice to avoid repeating the whole list of build strings in the host and run sections.

As you suggested in the comments, one option is to define the build string in conda_build_config.yaml:

# conda_build_config.yaml
gdal_build:
  - py36h02fde04_1  # [osx and py==36]
  - py37h622575a_1  # [osx and py==37]
  - py38h57202bd_1  # [osx and py==38]
  - py36hbb8311d_1  # [linux and py==36]
  - py37hf8c3989_1  # [linux and py==37]
  - py38hfe926b7_1  # [linux and py==38]

# meta.yaml
requirements:
  build:
    - python
    - 
  host:
    - python
    - pip
    - numpy
    - gdal  2.4.4 {{ gdal_build }}

  run:
    - python
    - gdal  2.4.4 {{ gdal_build }}

Another option is to define a lookup table in a jinja variable, directly in meta.yaml. This is slightly uglier, perhaps, but at least all of logic is contained in a single file. I'm not sure which to prefer.

{% set platform = 'linux' if linux else 'osx' if osx else 'win' %}

{%
  set gdal_builds = {
    'osx': {
      36: 'py36h02fde04_1',
      37: 'py37h622575a_1',
      38: 'py38h57202bd_1',
    },
    'linux': {
      36: 'py36hbb8311d_1',
      37: 'py37hf8c3989_1',
      38: 'py38hfe926b7_1',
    }
  }
%}

requirements:
  build:
    - python
    - 
  host:
    - python
    - pip
    - numpy
    - gdal  2.4.4 {{ gdal_builds[platform][py] }}

  run:
    - python
    - gdal  2.4.4 {{ gdal_builds[platform][py] }}