Managing log files created by cron jobs

I have a cron job that copies its log file daily to my home folder.

Everyday it overrides the existing file in the destination folder, which is expected. I want to preserve the log from previous dates so that next time it copies the file to destination folder, it preserves the files from previous dates.

How do I do that?

The best way to manage cron logs is to have a wrapper around each job. The wrapper could do these things, at the minimum:

• initialize environment
• redirect stdout and stderr to log
• run the job
• perform checks to see if job succeeded or not
• send notifications if necessary
• clean up logs

Here is a bare bones version of a cron wrapper:

#!/bin/bash

log_dir=/tmp/cron_logs/$(date +'%Y%m%d')
mkdir -p "$log_dir" || { echo "Can't create log directory '$log_dir'"; exit 1; }

#
# we write to the same log each time
# this can be enhanced as per needs: one log per execution, one log per job per execution etc.
#
log_file=$log_dir/cron.log

#
# hitherto, both stdout and stderr end up in the log file
#
exec 2>&1 1>>"$log_file"

#
# Run the environment setup that is shared across all jobs.
# This can set up things like PATH etc. 
#
# Note: it is not a good practice to source in .profile or .bashrc here
#
source /path/to/setup_env.sh

#
# run the job
#
echo "$(date): starting cron, command=[$*]"
"$@"
echo "$(date): cron ended, exit code is $?"

Your cron command line would look like:

/path/to/cron_wrapper command ...

Once this is in place, we can have another job called cron_log_cleaner which can remove older logs. It's not a bad idea to call the log cleaner from the cron wrapper itself, at the end.

An example:

# run the cron job from command line
cron_wrapper 'echo step 1; sleep 5; echo step 2; sleep 10'

# inspect the log
cat /tmp/cron_logs/20170120/cron.log

The log would contain this after running the wrapped cron job:

Fri Jan 20 04:35:10 UTC 2017: starting cron, command=[echo step 1; sleep 5; echo step 2; sleep 10]
step 1
step 2
Fri Jan 20 04:35:25 UTC 2017: cron ended, exit code is 0