Timeout function using threading in python does not work

Solution 1:

A thread can not gracefully kill another thread, so with your current code, foo never terminates. (With thread.daemon = True the Python program will exit when only daemon threads are left, but that does not allow you to terminate foo without also terminating the main thread.)

Some people have tried to use signals to halt execution, but this may be unsafe in some cases.

If you can modify foo, there are many solutions possible. For instance, you could check for a threading.Event to break out of the while-loop.

But if you can not modify foo, you could run it in a subprocess using the multiprocessing module since unlike threads, subprocesses can be terminated. Here is an example of how that might look:

import time
import multiprocessing as mp

def foo(x = 1):
    cnt = 1
    while True:
        time.sleep(1)
        print(x, cnt)
        cnt += 1

def timeout(func, args = (), kwds = {}, timeout = 1, default = None):
    pool = mp.Pool(processes = 1)
    result = pool.apply_async(func, args = args, kwds = kwds)
    try:
        val = result.get(timeout = timeout)
    except mp.TimeoutError:
        pool.terminate()
        return default
    else:
        pool.close()
        pool.join()
        return val


if __name__ == '__main__':
    print(timeout(foo, kwds = {'x': 'Hi'}, timeout = 3, default = 'Bye'))
    print(timeout(foo, args = (2,), timeout = 2, default = 'Sayonara'))

yields

('Hi', 1)
('Hi', 2)
('Hi', 3)
Bye
(2, 1)
(2, 2)
Sayonara

Note that this has some limitations too.

• subprocesses receive a copy of the parent processes' variables. If you modify a variable in a subprocess, it will NOT affect the parent process. If your function func needs to modify variables, you will need to use a shared variable.

• arguments (passed through args) and keywords (kwds) must be picklable.

• processes are more resource-heavy than threads. Usually, you only want to create a multiprocessing Pool once at the beginning of a program. This timeout function creates a Pool every time you call it. This was necessary since we needed pool.terminate() to terminate foo. There might be a better way, but I haven't thought of it.

Solution 2:

You need to turn it into a daemon thread:

it = ...
it.daemon = True
it.start()

Otherwise it's created as a user thread, and the process won't get stopped until all the user threads have finished.

Note that with your implementation the thread will continue to run and consume resources even after you've timed out waiting for it. CPython's Global Interpreter Lock could exacerbate the issue further.

Solution 3:

The benefit of using multiprocessing is the processes do not share memory and whatever happens in the child remains limited to that function and will not cause other process to terminate. The easiest way to add a timeout of 3s to a child process is:

import multiprocessing

def my_child():
    function here

process = multiprocessing.Process(target=my_child)
process.daemon = True
process.start()

process.join(3)
if process.is_alive():
    process.terminate()