How can I repeat each test multiple times in a py.test run?
I want to run each selected py.test item an arbitrary number of times, sequentially.
I don't see any standard py.test mechanism for doing this.
I attempted to do this in the pytest_collection_modifyitems()
hook. I modified the list of items passed in, to specify each item more than once. The first execution of a test item works as expected, but that seems to cause some problems for my code.
Further, I would prefer to have a unique test item object for each run, as I use id (item) as a key in various reporting code. Unfortunately, I can't find any py.test code to duplicate a test item, copy.copy()
doesn't work, and copy.deepcopy()
gets an exception.
Can anybody suggest a strategy for executing a test multiple times?
One possible strategy is parameterizing the test in question, but not explicitly using the parameter.
For example:
@pytest.mark.parametrize('execution_number', range(5))
def run_multiple_times(execution_number):
assert True
The above test should run five times.
Check out the parametrization documentation: https://pytest.org/latest/parametrize.html
The pytest module pytest-repeat exists for this purpose, and I recommend using modules where possible, rather than re-implementing their functionality yourself.
To use it simply add pytest-repeat
to your requirements.txt
or pip install pytest-repeat
, then execute your tests with --count n
.