How to keep cron from stampeding?
Say I have several cron scripts that need to run every 15 minutes. I could set them to run: */15 * * * *
, but then they all run at the same time. It seems silly for the server to sit idle for several minutes and then suddenly try to execute a dozen scripts all at the same time.
Is there a way I can have one script run at minute 1, 16, 31, 46 and another at 2, 17, 32, 47?
In other words, I want each script to run every 15 minutes, but I don't care that they run specifically on the quarter hour marks.
Solution 1:
You're making this harder than it needs to be. Put them all on the same line, separated by semicolons:
*/15 * * * * command1 ; command 2 ; command 3
It will run command1
, wait for it to finish, then run command2
, wait for it to finish, and so on.
Solution 2:
If you make your cron job look like this: 6-59/15 * * * *
then it will run at 6, 21, 36 and 51 minutes past the hour.
This may not work with all versions of cron.
Solution 3:
You could put all of the scripts into a directory, say /etc/cron.15m
, and then have cron run
*/15 * * * * run-parts /etc/cron.15m
That's assuming you have the run-parts
command. It's present on all Debian-based systems at least. It runs all of the executable programs in the named directory, one at a time in list order.
A disadvantage of this method is that if one of the scripts hangs, all the rest will wait and not be executed. If the run time of all of them is more than 15 minutes, then the job will start running again and you could get a lot of processes piling up.