Cron only runs first part of script [duplicate]

Solution 1:

The command is getting truncated because % has a special meaning in cron. From man 5 crontab:

   The ``sixth'' field (the rest of the line) specifies the command to  be
   run.   The  entire  command  portion  of the line, up to a newline or %
   character, will be executed by /bin/sh or by the shell specified in the
   SHELL  variable of the crontab file.  Percent-signs (%) in the command,
   unless escaped with backslash (\), will be changed into newline charac‐
   ters,  and  all  data  after the first % will be sent to the command as
   standard input. There is no way to split a  single  command  line  onto
   multiple lines, like the shell's trailing "\".

which explains why the logged command looks something like:

CMD (find /home/steeldriver/forums/tests -type f -mtime +1 -printf ')

To demonstrate that escaping works, given:

$ ls -l tests
total 12
-rw-rw-r-- 1 steeldriver steeldriver  0 Jan 22 14:22 A8eVAmK.txt
-rw-rw-r-- 1 steeldriver steeldriver 22 Oct 27 22:18 A9E27.txt
-rw-rw-r-- 1 steeldriver steeldriver 10 Oct 27 22:19 ffn2eG6.txt
-rw-rw-r-- 1 steeldriver steeldriver 43 Jan 22 14:22 sBHFgkv.txt

Then adding the crontab as:

* * * * * find /home/steeldriver/forums/tests -type f -mtime +1 -printf '\%P\0' | rsync -0av --exclude-from=- /home/steeldriver/forums/tests/ /home/steeldriver/forums/newtests/ 2>&1 > /home/steeldriver/forums/backup.log

(note this uses the null-terminated form \0) produces the log file:

$ cat backup.log
sending incremental file list
created directory /home/steeldriver/forums/newtests
./
A8eVAmK.txt
sBHFgkv.txt

sent 250 bytes  received 113 bytes  726.00 bytes/sec
total size is 43  speedup is 0.12

indicating that the backup ran successfully and copied only the newer files:

$ ls -l newtests
total 4
-rw-rw-r-- 1 steeldriver steeldriver  0 Jan 22 14:22 A8eVAmK.txt
-rw-rw-r-- 1 steeldriver steeldriver 43 Jan 22 14:22 sBHFgkv.txt

Note that you won't be able to test the escaping outside of your crontab - in the terminal, it will result in an error from find.

Solution 2:

I suggest an alternate approach. Using temporary files can make your life easier and will help you avoid using pipes which can cause issues with cron jobs.

The process here is fairly straightforward. First issue the command

find ./ -daystart -mtime +2 >myfile

Note:

The above will match files between 2 and 3 days old as it uses integer days.* Modify as necessary or use -mmin and minutes for more accurate selection. The matching results will be written to the file myfile which will be` overwritten every time the command is run.

Next issue the command: sed 's|./||' myfile >EXCL_list to strip off the unneeded prefix. The file EXCL_list will be`overwritten every time the command is run.

Finally issue the command:

rsync -Pav --exclude-from=EXCL_list --delete-excluded source/ dest

Explanation of rsync switches:

-P same as --partial --progress

-a archive mode

-v verbose

--exclude-from=EXCL_list read exclude patterns from FILE

--delete-excluded also delete excluded files from dest dirs

for more in depth information and further options see man rsync

Assumptions:

Commands are run in the source directory that contains the files and folders you want to process.

dest in the rsync command is a mounted destination that you have permission to write to.

Here's hoping this helps.

Sources:

http://www.linuxquestions.org/questions/linux-general-1/rsync-only-60-day-old-files-580357

https://unix.stackexchange.com/questions/92346/why-does-find-mtime-1-only-return-files-older-than-2-days

https://serverfault.com/questions/279609/what-exactly-will-delete-excluded-do-for-rsync

man find

man rsync