How should I deal with logs before they get to big?

I found out my disk was full today, and I freed up 10% just from deleting mostly logs from multiple websites. I, personally, like to keep logs as an archive, because I may need to look back on them for whatever reason. I'm new to administrating my LAMP server, so I want to pick up some good practices. How periodically should I clean out my logs? Should I even keep them? What other things can I do periodically to clean out the caches and "temporary" files for my server to keep it from using up all the diskspace again?

Solution 1:

There is a tool called logrotate which is designed to do exactly what you want to do by hand. Depending on how you configure it, it will e.g. once a day create a new log file and renames to old one to logfile.1, logfile.2 etc. And the oldest one gets then deleted.

Depending on your situation you maybe want to backup the log files to another server as well.

Here is another how-to about the tool.

Solution 2:

Yes, use logrotate to deal with logs.

You may also want to consider using a monitoring tool like monit to warn you of critical conditions in disk space.

Solution 3:

Use logrotate - which should already be on your debian system by default.

You may simply need to adjust the policies to catch logs that aren't currently being rotated, or adjust the retention policies to keep the archived copies less long.

Check /etc/logrotate.conf and the files in /etc/logrotate.d/ - if you installed Apache via the debian package, there should already be a policy in there for it that simply needs to be pointed at your logs.

Solution 4:

Apart from using logrotate you could also use the apache-httpd means rotatelogs. But your question aims into a different area, too.

There are propably two reasons to keep (http) logs:

• Statistics (and other lies)
• Keeping track of error conditions

For decent statistic visualization there is AWStats. A program that analyzes log-files and notifies about important things is logwatch.

If you have both in place you can rotate your logs away after some days.