Is there a way to tell pytest which tests to run based on a marker attribute value?

I have a test separated by a version using markers:

Example:

@pytest.mark.version("1.0")
def test_does_something():
     assert 1

@pytest.mark.version("1.0.1")
def test_does_another_thing():
     assert 0

@pytest.mark.version("1.1.1")
def test_does_nothing():
     assert 0

@pytest.mark.version("2.3.1")
def test_example():
     assert 1

The idea is to execute tests based on the passed by command line mark value like:

pytest my_file.py -m "version("1.1") and version("2.3.1")

Solution 1:

We can have a post collection filter as described here:
https://docs.pytest.org/en/latest/writing_plugins.html

This filter removes all tests where either version is not specified or it's not matching.

def pytest_collection_modifyitems(session, config, items):
    markerExpr = config.option.markexpr
    if not markerExpr:
        return
    tokens = markerExpr.split("version=")
    if len(tokens) == 0:
        return
    version = tokens[1]
    selected = []
    deselected = []
    for item in items:
        versionMarker = item.get_closest_marker('version')
        print(versionMarker)
        if not versionMarker:
            deselected.append(item)
            continue
        found = False
        if len(versionMarker.args) == 1:
            if versionMarker.args[0] == version:
                found = True
                selected.append(item)
        if not found:
            deselected.append(item)
    config.option.markexpr = "version"
    config.hook.pytest_deselected(items=deselected)
    items[:] = selected

How to use this?

pytest -v -m "version=1.1.1" 

t.py

import pytest


@pytest.mark.version("1.0")
def test_does_something():
     assert 1

@pytest.mark.version("1.0.1")
def test_does_another_thing():
     assert 0

@pytest.mark.version("1.1.1")
def test_does_nothing():
     assert 0

@pytest.mark.version("2.3.1")
def test_example():
     assert 1

Solution 2:

Following the example given in the documentation, you may use a conftest.py file to configure the mark properly:

# content of conftest.py

import pytest

def pytest_addoption(parser):
    parser.addoption(
        "-V",
        action="store",
        metavar="VERSION",
        help="only run tests matching the version VERSION.",
    )

def pytest_configure(config):
    # register an additional marker
    config.addinivalue_line(
        "markers", "version(x): mark test to run only on x version"
    )

def pytest_runtest_setup(item):
    versions = [mark.args[0] for mark in item.iter_markers(name="version")]
    print('versions: %s' % versions)
    print('item: %s' % item)
    if versions:
        if item.config.getoption("-V") not in versions:
            pytest.skip("test requires version in {!r}".format(versions))

Use the following command to run it (eg. only version 1.0.1):

pytest -V 1.0.1

It will print, among other things:

...
collected 4 items                                                                                                                                                                                             
test_mark.py sFss
...
1 failed, 3 skipped in 0.09s