Crontab not executing a Python script? [duplicate]

My python script is not running under my crontab.

I have placed this in the python script at the top:

#!/usr/bin/python

I have tried doing this:

chmod a+x myscript.py

Added to my crontab -e:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=""

* * * * * /home/me/project/myscript.py

My /var/log/cron file says:

Sep 21 11:53:02 163-dhcp /USR/SBIN/CROND[2489]: (me) CMD (/home/me/project/myscript.py)

But my script is not running because when I check my sql database, nothing has changed. If I run it directly in the terminal like so:

python /home/me/project/myscript.py

I get the correct result.

This is the myscript.py:

#!/usr/bin/python

import sqlite3

def main():
    con = sqlite3.connect("test.db")

    with con:

        cur = con.cursor()

        cur.execute("CREATE TABLE IF NOT EXISTS testtable(Id INTEGER PRIMARY KEY, Name TEXT)")

        cur.execute("INSERT INTO testtable(Name) VALUES ('BoB')")

        cur.execute("SELECT * FROM testtable")

        print cur.fetchall()    

if __name__ == "__main__":
    main()

Per comments: Yes, /usr/bin/python exists. I can also run the python script directly using just /home/me/project/myscript.py. /usr/bin/python /home/me/project/myscript.py works. So I don't believe this is the cause?

Solution 1:

There are a lot of half answers across the internet so I thought I would capture this to save someone else some time.

First, cronjob does a poor job of telling you where this is failing. I recommend sending stderr output to a log file like this:

Crontab Command:

# m h  dom mon dow   command
* * * * * /path/to/your_file.sh >> out.txt  2>&1

As this is likely running the command as user, check home directory for the log file. Note this script runs every minute which is good for debugging.

The next issue is you probably have a path problem... as script likely is trying to execute from your home directory. This script sets the current directory, echos it to file, and then runs your program.

Try this :

Script File

#!/bin/sh
cd "$(dirname "$0")";
CWD="$(pwd)"
echo $CWD
python your_python_file.py

Hope this saves someone else some debugging time!!!

Solution 2:

What happens when you type

/home/me/project/myscript.py into the shell?

Can you explicitly use /usr/bin/python in your crontbb command?

Can you either use an absolute path to your test.db or cd to the correct directory then execute your python script?

This is helpful to have debug statements in your python and log some data. Crontab can be very tricky to debug.

Solution 3:

It is possible that the script does not start because it cannot locate the python interpreter. Crontab environment may be very different from the shell environment which you are using. The search paths may be differ significantly. Also, you test your script by starting the python interpreter explicitly while you expect the crontab to only start the script. I put this line at the top of my python scripts:

\#!/bin/env python

This line will help locate the interpreter regardless of which directory it is installed in as long as it is in the search path.

Solution 4:

It's usually because the python used by crontab is different from the one you use in the shell. The easiest way to solve this is:

1. get the python you use in the shell:

$ which python # it may be "python3" or something else /usr/bin/python

2. use that specific python in crontab file:

* * * * * /usr/bin/python test.py

Also want to mention that using env -i /bin/bash --noprofile --norc in the shell lets you have the same environment as the one used by crontab, and this is super helpful to debug.

Solution 5:

Typically, crontab problems like this are caused by the PATH environment variable being more restrictive/different than what your normal user's PATH environment is. Since your shell uses the PATH environment to find the executable (e.g. /usr/bin/python is found in /usr/bin when you type "python" at a shell prompt), when the PATH is missing common locations, like /usr/bin or /usr/sbin, your cron job will fail. This has bit me many times. The simple fix is just to explicitly set the PATH yourself near the top of your crontab file, before any commands that need it. So, just edit the crontab as usual and add something like this near the top (if your binary is not in one of the below paths, you'll need to add it after a colon):

PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

Alternately, just use absolute paths to your binaries and scripts in crontab.