How to run celery on windows?

How to run celery worker on Windows without creating Windows Service? Is there any analogy to $ celery -A your_application worker?

Solution 1:

Celery 4.0+ does not officially support window already. But it still works on window for some development/test purpose.

Use eventlet instead as below:

pip install eventlet
celery -A <module> worker -l info -P eventlet

It works for me on window 10 + celery 4.1 + python 3.

This solution solved the following exception:

[2017-11-16 21:19:46,938: ERROR/MainProcess] Task handler raised error: ValueError('need more than 0 values to unpack',)
Traceback (most recent call last):
  File "c:\users\wchen8\work\venv\weinsta\lib\site-packages\billiard\pool.py", line 358, in workloop
    result = (True, prepare_result(fun(*args, **kwargs)))
  File "c:\users\wchen8\work\venv\weinsta\lib\site-packages\celery\app\trace.py", line 525, in _fast_trace_task
    tasks, accept, hostname = _loc
ValueError: need more than 0 values to unpack

===== update 2018-11 =====

Eventlet has an issue on subprocess.CalledProcessError:

https://github.com/celery/celery/issues/4063

https://github.com/eventlet/eventlet/issues/357

https://github.com/eventlet/eventlet/issues/413

So try gevent instead.

pip install gevent
celery -A <module> worker -l info -P gevent

This works for me on window 10 + celery 4.2 + python 3.6

Solution 2:

yes:

celery -A your_application -l info

also note Celery have dropped support for Windows(since v4), so best to

pip install celery==3.1.25

3.1.25 was the last version that works on windows(just tested on my win10 machine). Didn't need to downgrade flower(browser monitor for celery) though.

See also the FAQ for Windows

Solution 3:

There are two workarounds to make Celery 4 work on Windows:

• use eventlet, gevent or solo concurrency pool (if your tasks as I/O and not CPU-bound)
• set the environment variable FORKED_BY_MULTIPROCESSING=1 (this is what actually causes the underlying billiard package to to fail under Windows since version 4)

See https://www.distributedpython.com/2018/08/21/celery-4-windows for more details

Solution 4:

I have run celery task using RabbitMQ server. RabbitMq is better and simple than redis broker

while running celery use this command "celery -A project-name worker --pool=solo -l info" and avoid this command "celery -A project-name worker --loglevel info"

Solution 5:

It's done the same way as in Linux. Changing directory to module containing celery task and calling "c:\python\python" -m celery -A module.celery worker worked well.