How many cron jobs are too many?

I have a couple of cron jobs for basic maintenance which aren't very resource-intensive.

I also have custom task scheduling (which is just calling a .php file and passing information via GET, ie: cronjob.php?param1=param ...). These can add up pretty quickly.

These just call system commands and run external programs (Nmap is one of them). They usually don't take long either.

Anyway, can anyone tell me, roughly what point is too many? I know it's hard to say since it depends on what job is being run and how often, but at what point does the crontab program start "struggling"? Anyone have any idea?

Thanks.

Solution 1:

I wouldn't worry about the load on the "crontab program" (cron) itself; it's your overall system load you might want to pay attention to. Look at metrics (cpu utilization, io rates, web query response times) during the time your job(s) are running - is there a noticeable spike? is it bad enough that it's disrupting actual use of the system?

If the programs "don't take long", that's a good sign that it's not a problem.

If you're still concerned, you can do other things to limit the load: run the jobs with nice to reduce their priority, run them sequentially instead of simultaneously, and so forth.

Solution 2:

Just searching through our production systems, the largest one had 862 cron jobs (across all users, the largest single one, root, is 117), and it doesn't particularly break out in a sweat with that many.

If you start 60 jobs every minute, 24x7, and they all take two seconds to complete, then you'll probably end up in a world of pain. But, as long as the workload balances out (with quiet times and such), it should recover. Keep an eye on system loads during the peak times (CPU%, disk I/O, ...) and worry more about that than the number of actual jobs.

This question is akin to "How many processes can I run before getting into trouble?". The best bet is to try it, since it depends on your system. Cron itself has few limitations that I've found but, if you're trying to schedule tens of thousands of jobs, you're putting more of a strain on it than I ever had.

Measure, don't guess!

Solution 3:

Cron itself will essentially never start to struggle; the system will be completely bogged down with workload long before that happens.