Linux: logwatch(8) is too noisy. How can I control the noise level?

Solution 1:

Overall, the available documentation for Logwatch lacks adequate explanation and is often far too vague. I pieced together some useful examples, and have reduced the Logwatch noise by over 95%.

Here's what I have found.

Keep in mind that you can find some Logwatch documentation at /usr/share/doc/logwatch-*/HOWTO-Customize-LogWatch, and it contains a few useful examples.

1. On RHEL/CentOS/SL, the default logwatch configuration is under /usr/share/logwatch/default.conf/logwatch.conf

   These settings can be overriden by placing your local configuration under /etc/logwatch/conf/logwatch.conf. Place the following in that file to tell logwatch to completely ignore services like 'httpd' and the daily disk usage checks:

   # Don't spam about the following Services
   Service = "-http"
   Service = "-zz-disk_space"
   

2. Sometimes I don't want to completely disable logwatch for a specific service, I just want to fine tune the results to make them less noisy. /usr/share/logwatch/default.conf/services/*.conf contains the default configuration for the services. These parameters can be overridden by placing your local configuration under /etc/logwatch/conf/services/$SERVICE.conf. Unfortunately, logwatch's ability here is limited, and many of the logwatch executables are full of undocumented Perl. Your choice is to replace the executable with something else, or try to override some settings using /etc/logwatch/conf/services.

   For example, I have a security scanner which runs scans across the network. As the tests run, the security scanner generates many error messages in the application logs. I would like logwatch to ignore errors from my security scanners, but still notify me of attacks from other hosts. This is covered in more detail at Logwatch: Ignore certain IPs for SSH & PAM checks?. To do this, I place the following under /etc/logwatch/conf/services/sshd.conf:

   # Ignore these hosts
   *Remove = 192.168.100.1
   *Remove = X.Y.123.123
   # Ignore these usernames
   *Remove = testuser
   # Ignore other noise. Note that we need to escape the ()
   *Remove = "pam_succeed_if\(sshd:auth\): error retrieving information about user netscan.*
   

   "

3. logwatch also allows you to strip out output from the logwatch emails by placing regular expressions in /etc/logwatch/conf/ignore.conf. HOWTO-Customize-LogWatch says:

   ignore.conf: This file specifies regular expressions that, when matched by the output of logwatch, will suppress the matching line, regardless of which service is being executed.

   However, I haven't had much luck with this. My requirements need a conditional statement, which is something like 'If there are security warnings due to my security scanner, then don't print the output. But if there are security warnings from my security scanner and from some bad guys, then print the useful parts-- The header which says "Failed logins from:", the IPs of the bad hosts, but not the IPs of scanners.'

4. Nip it at the source (As suggested by @user48838). These messages are being generated by some application, and then Logwatch is happily spewing the results to you. In these cases, you can modify the application to log less.

   This isn't always desirable, because sometimes you want the full logs to be sent somewhere (to a Central syslog server, central IDS server, Splunk, Nagios, etc.), but you don't want logwatch to email you about this from every server, every day.

Solution 2:

Yes - logwatch is often too noisy. You already mentioned disabling checks completely.

If you don`t want to do that you have to prevent certain events to appear. For instance - it is not interesting if nagios connects via ssh to a DMZ system. But is is interesting if there are other logins via ssh.

We use rsyslog instead of ksyslogd (first install rsyslog, then remove ksyslogd). With rsyslog you can fine-tune what goes to the logs and what not (e.g. build a expression that drops messages from sshd cointaining "nagios connected"). That way logwatch will report useful information only.

Another case might be xinetd - I don`t want to get informed about successful connects - this can be configured in xinetd itselv - without disabling the logwatch-check for xinetd.

Solution 3:

As a point of interest I followed option 2 form the answer by Stefan Lasiewski but for my purposes I wanted to only include specific lines rather than exclude all the noise I didn't want.

I was configuring vsftpd so I created /etc/logwatch/conf/services/vsftpd.conf and instead of using something like *Remove = testuser which removes rows which include the text testuser I used the line *OnlyContains = "testuser" which only returns rows including that text.

These 2 scripts work very basically by using grep and grep -v.

The difference being that you can use *Remove as many times as you want but with *OnlyContains you have to use it once if you want something or something else or something else. So for multiple values you do *OnlyContains = "testuser|testuser2|testuser3"