How I can monitor new files in a directory and move/rename them into another directory?

A program generates output textfiles, named output.txt, after every 15th iteration of a certain process. Doing so, it overwrites the last output.txt. I want to keep the files however, and I can't modify the file name within the program.

Can I run some script, together with the program, that monitors the output directory and moves and renames the output.txt files into another directory?

Solution 1:

First install the package inotify-tools:

sudo apt-get install inotify-tools

A bash script would help

#! /bin/bash

folder=~/Desktop/abc

cdate=$(date +"%Y-%m-%d-%H:%M")

inotifywait -m -q -e create -r --format '%:e %w%f' $folder | while read file

  do
    mv ~/Desktop/abc/output.txt ~/Desktop/Old_abc/${cdate}-output.txt
  done

What does this script means:

This will watch the folder ~/Desktop/abc so whenever a file created inside then it's going to move the file founded inside which is name output.txt to a directory ~/Desktop/Old_abc and rename providing a suffix of date and time of t he new file, this to be sure not to overwrite old files and like that you can also know this file was created in what time and date

Solution 2:

The script below will move and rename any file that might appear in a defined directory (dr1). It renames the files like: output_1.txt, output_2.txt` etc. The script looks "actively" if the targeted name already exists in directory 2 (not from a "blindly" chosen range), so you can start and stop the script at any time without the risk of overwriting existing files.

Since it gives the output files a unique name, and does by definition not overwrite existing files, the targeted directory can be the same as the source directory.

How to use

Copy the script into an empty file, save it as rename_save.py
Important step: in the head section, set the time interval to check for new files. make sure the time interval is (much) shorter than the interval in which new files appear (the time 15 iterations take) or else new files will be created before the last one is moved.
Also in the head section, set the paths to both the source directory and the directory you want to save the renamed files to.
Run it by the command:
```
python3 /path/to/rename_save.py
```
while the other (iterating) script is running

The script

#!/usr/bin/env python3
import shutil
import os
import time

#--- set the time interval to check for new files (in seconds) below 
#    this interval should be smaller than the interval new files appear!
t = 1
#--- set the source directory (dr1) and target directory (dr2)
dr1 = "/path/to/source_directory"
dr2 = "/path/to/target_directory"

name = "output_"; extension = ".txt"
newname = lambda n: dr2+"/"+name+str(n)+extension

while True:
    newfiles = os.listdir(dr1)
    for file in newfiles:
        source = dr1+"/"+file
        n = 1; target = newname(n)
        while os.path.exists(target):
            n = n+1; target = newname(n)
        shutil.move(source, target)
    time.sleep(t)

Solution 3:

Install the package inoticoming
```
sudo apt-get install inoticoming
```

Create wrapper script watch_output:

#!/bin/bash
backup_folder="$HOME/backups"

filename="$1"

mkdir -p "$backup_folder"

if [ "$filename" == "output.txt" ]
then
    echo "New or changed file \"output.txt\" in $2"
    mv "$2/$filename" "$backup_folder/${filename%.*}.$(date +'%Y-%m-%d_%H:%M:%S').${filename##*.}"
fi

Make it executable:

chmod +x <full_path_of_watch_output_script>

Watch your folder output folder:

inoticoming "$HOME/output" <full_path_of_watch_output_script> {} "$HOME/output" \;

Example:

$ killall inoticoming
$ inoticoming "$HOME/output" ./watch_output {} "$HOME/output" \;
$ touch $HOME/output/output.txt
$ ls -log $HOME/backups
total 0
-rw-rw-r-- 1 0 Mai 13 14:32 output.2015-05-13_14:32:10.txt