Executing a function in the background while using limited number of cores/threads and queue the extra executions?
Solution 1:
the program crashes randomly. I guess
It would be easier to concentrate on one problem at a time, without guessing, so, what's the crash?
Here's a test with threading
that might inspire you, based on the example for queue
.
#!python3
#coding=utf-8
""" https://stackoverflow.com/q/49081260/ """
import sys, time, threading, queue
print(sys.version)
class myClass:
""" """
def __init__(self):
""" """
self.q = queue.Queue()
self.threads = []
self.num_worker_threads = 2
def backgroundFunc(self):
""" """
print("thread started")
while True:
item = self.q.get()
if item is None:
#self.q.task_done()
break
print("working on ", item)
time.sleep(0.5)
self.q.task_done()
print("thread stopping")
def mainFunc(self):
""" """
print("starting thread(s)")
for i in range(self.num_worker_threads):
t = threading.Thread(target=self.backgroundFunc)
t.start()
self.threads.append(t)
print("giving thread(s) some work")
for item in range(5):
self.q.put(item)
print("giving thread(s) more work")
for item in range(5,10):
self.q.put(item)
# block until all tasks are done
print("waiting for thread(s) to finish")
self.q.join()
# stop workers
print("stopping thread(s)")
for i in range(self.num_worker_threads):
self.q.put(None)
for t in self.threads:
self.q.join()
print("finished")
if __name__ == "__main__":
print("instance")
myClassInstance = myClass()
print("run")
myClassInstance.mainFunc()
print("end.")
It prints
3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit (Intel)]
instance
run
starting thread(s)
thread started
thread started
giving thread(s) some work
giving thread(s) more work
waiting for thread(s) to finish
working on 0
working on 1
working on 2
working on 3
working on 4
working on 5
working on 6
working on 7
working on 8
working on 9
stopping thread(s)
thread stopping
thread stopping
finished
end.