Test case execution order in pytest

Solution 1:

In general you can configure the behavior of basically any part of pytest using its well-specified hooks.

In your case, you want the "pytest_collection_modifyitems" hook, which lets you re-order collected tests in place.

That said, it does seem like ordering your tests should be easier -- this is Python after all! So I wrote a plugin for ordering tests: "pytest-ordering". Check out the docs or install it from pypi. Right now I recommend using @pytest.mark.first and @pytest.mark.second, or one of the @pytest.mark.order# markers, but I have some ideas about more useful APIs. Suggestions welcome :)

Edit: pytest-ordering seems abandoned at the moment, you can also check out pytest-order (a fork of the original project by the author).

Edit2: In pytest-order, only one marker (order) is supported, and the mentioned examples would read @pytest.mark.order("first"), @pytest.mark.order("second"), or @pytest.mark.order(#) (with # being any number).

Solution 2:

Maybe you can consider using dependency pytest plugin, where you can set the test dependencies easily.

Be careful - the comments suggest this does not work for everyone.

@pytest.mark.dependency()
def test_long():
    pass

@pytest.mark.dependency(depends=['test_long'])
def test_short():
    pass

This way test_short will only execute if test_long is success and force the execution sequence as well.

Solution 3:

There's also a plugin pytest-ordering that seems to meet your requirements.

Solution 4:

Pytest's fixtures can be used to order tests in a similar way they order the creation of fixtures. While this is unconventional, it takes advantage of knowledge you may already have of the fixture system, it does not require a separate package and is unlikely to be altered by pytest plugins.

@pytest.fixture(scope='session')
def test_A():
    pass

@pytest.mark.usefixtures('test_A')
def test_B():
    pass

The scope prevents multiple calls to test_A if there are multiple tests that depend on it.

Solution 5:

It's important to keep in mind, while trying to fix pytest ordering "issue", that running tests in the same order as they are specified seems to be the default behavior of pytest.

It turns out that my tests were out of that order because of one of these packages - pytest-dependency, pytest-depends, pytest-order. Once I uninstalled them all with pip uninstall package_name, the problem was gone. Looks like they have side effects