Limit the size of a directory by deleting old files

I have a IP cam which save its recordings in a specific directory named Camera1 in my Ubuntu Server 12.04.

I would like to limit the size of this folder to 5 gigs, by deleting -say once a day- the oldest files.

I first checked the quota program but it doesn't seem to allow the creation of new files and deleting of the old ones.

So I think the best method would be to run a bash script...

Solution 1:

I was in a very similar situation. I have a big LVM partition holding 2 directories:

• Videos (live streams recorded) can use 88% of the partition

• Pictures (pictures sent by the cameras when detecting motion) can use 7% of the partition (as pictures are much lighter, I can still keep older pictures than videos)

The remaining 5% is a security margin so that the partition never gets full. Note I am talking in percentages instead of fixed gigabytes because the size of my LVM partition changes if I replace or add a hard drive.

So here is the script (I added many comments so it's easily understandable):

#!/bin/bash
#Usage = sh limit-directory-size.sh /media/computer/mypartition 88 120 ("/media/computer/mypartition" = the directory to be limited / "88"=the percentage of the total partition this directory is allowed to use / "120"=the number of files to be deleted every time the script loops (while $Directory_Percentage > $Max_Directory_Percentage)

#Directory to limit
Watched_Directory=$1
echo "Directory to limit="$Watched_Directory

#Percentage of partition this directory is allowed to use
Max_Directory_Percentage=$2
echo "Percentage of partition this directory is allowed to use="$Max_Directory_Percentage

#Current size of this directory
Directory_Size=$( du -sk "$Watched_Directory" | cut -f1 )
echo "Current size of this directory="$Directory_Size

#Total space of the partition = Used+Available
Disk_Size=$(( $(df $Watched_Directory | tail -n 1 | awk '{print $3}')+$(df $Watched_Directory | tail -n 1 | awk '{print $4}') ))       
echo "Total space of the partition="$Disk_Size

#Curent percentage used by the directory
Directory_Percentage=$(echo "scale=2;100*$Directory_Size/$Disk_Size+0.5" | bc | awk '{printf("%d\n",$1 + 0.5)}')
echo "Curent percentage used by the directory="$Directory_Percentage

#number of files to be deleted every time the script loops (can be set to "1" if you want to be very accurate but the script is slower)
Number_Files_Deleted_Each_Loop=$3
echo "number of files to be deleted every time the script loops="$Number_Files_Deleted_Each_Loop

#While the current percentage is higher than allowed percentage, we delete the oldest files
while [ $Directory_Percentage -gt $Max_Directory_Percentage ] ; do
    #we delete the files
    find $Watched_Directory -type f -printf "%T@ %p\n" | sort -nr | tail -$Number_Files_Deleted_Each_Loop | cut -d' ' -f 2- | xargs rm
    #we delete the empty directories
    find $Watched_Directory -type d -empty -delete
    #we re-calculate $Directory_Percentage
    Directory_Size=$( du -sk "$Watched_Directory" | cut -f1 )
    Directory_Percentage=$(echo "scale=2;100*$Directory_Size/$Disk_Size+0.5" | bc | awk '{printf("%d\n",$1 + 0.5)}')
done

Then you call it every 15 minutes with a crontab entry

*/15 * * * * sh /home/computer/limit-directory-size.sh /media/computer/mypartition/videos 88 120

Hope it helps!

Solution 2:

I started thinking about how hard it would be to only keep a certain amount of files. I turned to awk, which I have not used for a while, and came up with the following one liner.

cd /path/to/Camera1 && ls -ltc | awk '{ if (!system("test -f " $9)) { size += $5; if (size > 5*2^30 ) system("rm " $9) } }'

1. change to directory in question
2. list files, newest first
3. Run awk on output, check that it is a regular file, add file size to counter, remove file if cumulative size over 5 gigs

You can change "rm " to "ls " to have it list the files it would remove. It would be insane not to carefully test a script, suggested by an unknown on the net, which removes files!

The script will probably break and/or not do what you expect if you have funny characters (space for example) in the filenames.