Multiprocessing + Flask OSError: [Errno 48] Address already in use / OSError: [Errno 38] Socket operation on non-socket
I am trying to build a Flask app that runs a function that eventually utilizes the multiprocessing library.
import time
from flask import Flask, request
import multiprocessing as mp
app = Flask(__name__)
def proc():
time.sleep(10)
@app.route('/greeting')
def greeting():
p = mp.Process(target=proc, args=())
p.start()
return ('Hello there')
app.run(port=8000)
The problem occurs when I go to http://localhost:8000/greeting
and get the following error trying to call p.start()
.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
prepare(preparation_data)
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/michaelequi/Github/drlm/apps/mp_flask.py", line 18, in <module>
app.run(port=8000)
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/flask/app.py", line 920, in run
run_simple(t.cast(str, host), port, self, **options)
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 1010, in run_simple
inner()
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 950, in inner
srv = make_server(
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 782, in make_server
return ThreadedWSGIServer(
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 688, in __init__
super().__init__(server_address, handler) # type: ignore
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socketserver.py", line 452, in __init__
self.server_bind()
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/server.py", line 138, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socketserver.py", line 466, in server_bind
self.socket.bind(self.server_address)
OSError: [Errno 48] Address already in use
If I change the code such that debug=True (app.run(port=8000, debug=True)
) I get a different error!
Traceback (most recent call last):
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 950, in inner
srv = make_server(
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 782, in make_server
return ThreadedWSGIServer(
File "/Users/michaelequi/Github/drlm/venv/drlm/lib/python3.9/site-packages/werkzeug/serving.py", line 676, in __init__
real_sock = socket.fromfd(fd, self.address_family, socket.SOCK_STREAM)
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 545, in fromfd
return socket(family, type, proto, nfd)
File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 232, in __init__
_socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 38] Socket operation on non-socket
This occurs no matter what port I run the flask server on. Anyone know what may be causing this and how to fix it? I am using Python 3.9.9 on MacOS Big Sur.
Replace
app.run(port=8000)
with
if __name__ == '__main__':
app.run(port=8000)
When using multiprocessing, anything that is "top level" gets run in every thread. So both your main thread and the thread you're creating try to open port 8000