How to retry after exception?
I have a loop starting with for i in range(0, 100)
. Normally it runs correctly, but sometimes it fails due to network conditions. Currently I have it set so that on failure, it will continue
in the except clause (continue on to the next number for i
).
Is it possible for me to reassign the same number to i
and run through the failed iteration of the loop again?
Do a while True
inside your for loop, put your try
code inside, and break from that while
loop only when your code succeeds.
for i in range(0,100):
while True:
try:
# do stuff
except SomeSpecificException:
continue
break
I prefer to limit the number of retries, so that if there's a problem with that specific item you will eventually continue onto the next one, thus:
for i in range(100):
for attempt in range(10):
try:
# do thing
except:
# perhaps reconnect, etc.
else:
break
else:
# we failed all the attempts - deal with the consequences.
UPDATE 2021-12-01:
As of June 2016, the retrying package is no longer being maintained. Consider using the active fork github.com/jd/tenacity, or alternatively github.com/litl/backoff.
The retrying package is a nice way to retry a block of code on failure.
For example:
@retry(wait_random_min=1000, wait_random_max=2000)
def wait_random_1_to_2_s():
print("Randomly wait 1 to 2 seconds between retries")
Here is a solution similar to others, but it will raise the exception if it doesn't succeed in the prescribed number or retries.
tries = 3
for i in range(tries):
try:
do_the_thing()
except KeyError as e:
if i < tries - 1: # i is zero indexed
continue
else:
raise
break
Alternatives to retrying
: tenacity
and backoff
(2020 update)
The retrying library was previously the way to go, but sadly it has some bugs and it hasn't got any updates since 2016. Other alternatives seem to be backoff and tenacity. During the time of writing this, the tenacity had more GItHub stars (2.3k vs 1.2k) and was updated more recently, hence I chose to use it. Here is an example:
from functools import partial
import random # producing random errors for this example
from tenacity import retry, stop_after_delay, wait_fixed, retry_if_exception_type
# Custom error type for this example
class CommunicationError(Exception):
pass
# Define shorthand decorator for the used settings.
retry_on_communication_error = partial(
retry,
stop=stop_after_delay(10), # max. 10 seconds wait.
wait=wait_fixed(0.4), # wait 400ms
retry=retry_if_exception_type(CommunicationError),
)()
@retry_on_communication_error
def do_something_unreliable(i):
if random.randint(1, 5) == 3:
print('Run#', i, 'Error occured. Retrying.')
raise CommunicationError()
for i in range(100):
do_something_unreliable(i)
The above code outputs something like:
Run# 3 Error occured. Retrying.
Run# 5 Error occured. Retrying.
Run# 6 Error occured. Retrying.
Run# 6 Error occured. Retrying.
Run# 10 Error occured. Retrying.
.
.
.
More settings for the tenacity.retry
are listed on the tenacity GitHub page.