cron job that fired 8 times rather than once. Daylight saving issue?
We're running a Redhat server (2.6.33.3-85.fc13.x86_64) and experienced some odd behaviour with our cron setup. We're running an out-of-the-box installation of cron.
We have a task that runs daily at 2am and has done so successfully for many months. However yesterday (27/03/2011) morning 8 instances of the task were started.
Here is the relevant section of our cron log file:
...
Mar 27 02:00:01 fs1 CROND[19557]: (root) CMD (true && /home/youdev/you_server/NEW_FS1/housekeeping_scripts/copy_pdf_printer_files_to_fs2.sh)
Mar 27 02:00:01 fs1 CROND[19563]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:00:02 fs1 CROND[19597]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:00:12 fs1 CROND[19931]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:00:22 fs1 CROND[20060]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:00:32 fs1 CROND[20067]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:00:42 fs1 CROND[20077]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:00:52 fs1 CROND[20084]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:01:02 fs1 CROND[20094]: (youdev) CMD (true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh)
Mar 27 02:01:02 fs1 CROND[20095]: (root) CMD (run-parts /etc/cron.hourly)
Mar 27 02:01:02 fs1 CROND[20096]: (root) CMD (true && /home/youdev/you_server/NEW_FS1/housekeeping_scripts/copy_pdf_printer_files_to_fs2.sh)
Mar 27 02:01:02 fs1 run-parts(/etc/cron.hourly)[20095]: starting 0anacron
...
Each of the fetch_db_data.sh scripts started correctly (they're just a shell script that calls a rsync - and the rsync tmp files exist as do the files that were successfully rsynced across).
This is the only job we have that starts at 02:00, yet we have a job that runs on the minute (copy_pdf_printer_files_to_fs2.sh) and this correctly ran just once at 02:00.
The only thing that we can think could have caused it is a daylight saving issue (we're in the UK so went from GMT to BST at 02:00 on the day in question) - however all the digging we've done says that cron handles daylight saving issues fine. And even then, shouldn't we have expected 0 runs (as time skipped from 00:59 to 02:00) rather than 8?
Here is the crontab entry:
[email protected]
# BACKUP LATEST DB FROM LIVE (takes approx one hour)
0 2 * * * true && /home/youdev/you_server/NEW_FS1/backup_scripts/fetch_db_data.sh
Many thanks in advance for reading this and for any help and suggestions,
Chris.
I'd use Ockham's Razor with this. It's bound to be the Summer Time clock changes.
Please see Daniël van Eeden answer to question Cron job executed multiple times daylight saving time:
It is a known bug:
http://bugs.centos.org/view.php?id=5590
https://bugzilla.redhat.com/show_bug.cgi?id=436694
I've seen it happen on CentOS 6.2 with cronie-1.4.4-7.el6.x86_64 on the 31st of march (CET -> CEST)