pytest run tests parallel

I want to run all my pytest tests in parallel instead of sequentially.

my current setup looks like:

class Test1(OtherClass):
    @pytest.mark.parametrize("activity_name", ["activity1", "activity2"])
    @pytest.mark.flaky(reruns=1)
    def test_1(self, activity_name, generate_test_id):
    """
    """

        test_id = generate_random_test_id()
        test_name = sys._getframe().f_code.co_name

        result_triggers = self.proxy(test_name, generate_test_id, test_id, activity_name)

        expected_items = ["response"]
        validate_response("triggers", result_triggers, expected_items)


    @pytest.mark.parametrize("activity_name", ["activity1", "activity2"])
    @pytest.mark.flaky(reruns=1)
    def test_2(self, activity_name, generate_test_id):
    """
    """

        #same idea...

I run my tests using pytest -v -s.

The result is that my tests are running sequentially, which takes a lot of time since some of them wait for responses from remote servers (integration tests).

Is there any way of running pytest in parallel?

Solution 1:

You want pytest-xdist. I think Qxf2 explains it quite well: Qxf2 on Pytest-Xdist

Their Linux command-line is slightly too verbose for my tastes though; I use:

pytest -n <NUM>

where <NUM> is the number of parallel workers.

Solution 2:

pytest-xdist is a great solution for most cases, but integration tests are special. After sending a request to a remote server, another test can start on a new thread instead of waiting for a response. This is concurrent testing instead of parallel. Concurrency allows many more tests at once with much less memory and processing overhead.

I wrote the pytest-parallel plugin [py3.6+] to enable parallel and concurrent testing. Here's how to run your integration tests concurrently:

pytest --tests-per-worker auto

Solution 3:

In order to run pytests in parallel you are going to need to install pytest-xdist. Please see the different parallelism tactics listed below, you can use any of those (however I can bet that one of those suits best for your particular case):

Examples

pip install pytest-xdist

# The most primitive case, sending tests to multiple CPUs:
pytest -n NUM

# Execute tests within 3 subprocesses.
pytest --dist=each --tx 3*popen//python=python3.6

# Execute tests in 3 forked subprocess. Won't work on windows.
pytest --dist=each --tx 3*popen//python=python3.6 --boxed

# Sending tests to the ssh slaves
pytest --dist=each --tx ssh=first_slave --tx ssh=seconds_slave --rsyncdir package package

# Sending tests to the socket server, link is available below.
python socketserver.py :8889 &
python socketserver.py :8890 &
pytest --dist=each --tx socket=localhost:8889 --tx socket=localhost:8890

You can provide different values for --dist(-d) parameter, which handles the way tests are distributed across workers, see documentation for more information regarding the usage of --dist.

NOTE: Once tests are executed the socket_server.py is down. I suggest you running socket server from separate terminal windows for debugging purposes

You can introduce more complicated flows, for instance running tests inside docker containers with have started socket servers kind of "pytest workers" and another docker container which communicates to them and serve as "pytest runner".

pytest run tests parallel

Solution 1:

Solution 2:

Solution 3:

Examples

Related

Recent Posts