Linux: how to send new lines in log files to remote syslog?

Solution 1:

You've already rejected "other people's bash scripts", but this is a pretty common solution -- some creative use of the logger command can follow a file and send its contents elsewhere.
I personally wouldn't do this in a production environment though.

A better option which requires less scripting hackery is using rsyslogd and the text file input module like yoonix mentioned -- This is a pretty decent solution though there is some potential for lost lines during a file rotation, and if you're on a Linux system with rsyslog as your syslog daemon there's not much additional work required.

syslog-ng also supports a file input source with functionality similar to rsyslog's.

IMHO the best solution - albeit one which requires modifying the application generating these logs - is to log to syslog directly. You don't want to be going through intermediary steps, files, etc. -- syslog is the SYStem LOGger, and things that write logs on a Unix platform should be sending them to syslog.
Implementation of this is, unfortunately, left as an exercise for the reader (and application developer) and may not be possible if your developers are nonexistent, lazy, or incompetent....

Solution 2:

You could use logstash with the file input and syslog output.

For example, create a configuration with the file (or files) you want to monitor and your syslog server info.

file-to-syslog.conf:

input { file { path => "/var/log/kern.log" } }
output {
    syslog {
        facility => "kernel"
        host => "syslog.example.com"
        port => 514
        severity => "informational"
    }
}

The start up logstash with

java -jar logstash-1.2.2-flatjar.jar agent -f file-to-syslog.conf

Solution 3:

I hacked together tail.c and logger.c into a single, small footprint compiled program (binary) that is lightweight, fast and stable. As long as it has read access to the log file(s), then it works without needing root privilege.

I also made a couple improvements to the native logger and added a new (optional) capability of inserting a text string at the beginning of each log line before it gets sent to the log server. The result is a program that can be run by itself, without needing to use shell pipes (i.e. don't need to tail logfile | logger). It will run forever until explicitly killed or it encounters an error writing to the network socket. It even continues to run if the log file is rotated or even disappears (it will just continue to look to see if the file reappears.)

It's easy to use: just give it one or more log file(s) to monitor, and each time a new line gets written to the file, it will send a copy of that line to the local or remote syslog server you specify. Plus the extra text string if you use that option.

I actually finished the program back in December, but was waiting for Yahoo to take copyright and make it available, which they've now done. (I wrote it as part of my job at Yahoo).

filelogger program information and download link:

• Blog post about filelogger
• GitHub Repo for filelogger

Solution 4:

There are a number of ways to tackle this. But the very, very first thing you should do is: forward the logs using syslog itself.

Syslog (and many replacements for syslog) have built-in facilities to forward logging to another syslog server at a different address. You can easily do so by changing the configuration file and appending the address to forward the facility to. For instance, adding this line to:

*.*    @192.168.1.1

...would forward all facilities to the machine at 192.168.1.1, which (hopefully) has the service running. The example I give here is for rsyslog, which is the stock syslog server on Debian, although it should work for many others. Consult the documentation for your implementation of syslog with man syslog and see what it says about "forwarding".

The remote syslog server can be anything you like. There are even products, like Splunk, which happily aggregate these logs into a single view with a web dashboard, search, event-driven notifications, etc. etc. You can see more here: http://www.splunk.com/ If that doesn't meet your needs, you can use something else. There are even syslog servers that will dump to a SQL database!

Sure, you could write your own script/program/service to do this for you, but why re-invent the wheel when it's both done for you and already given to you?

Edit: So I went back and re-read the question, and noticed several comments. It sounds like:

1. you want to aggregate your application logs
2. you don't have access to root
3. your application(s) just dump text somewhere
4. your application(s) doesn't know how to write to the local syslog
5. you don't have control over your application source code

So let's address each one in sequence:

1. syslog was meant to aggregate logs together. You can use anything you like, but there is a reason why it's been around for a long time. It's well-tested, well-debugged, well-documented, well-known, and for most *nix platforms nearly universally supported in one flavor or another.
2. we don't need access to root to set up logging. We only need access to the syslog API. root is not a requirement to write to the syslog; if this were the case, then all of those services that drop privileges would be unable to write diagnostics to the log files.

3. Re: text dumps, this is normal. however, you should be able to use a subshell to pipe the output of STDERR and STDOUT to a program that calls the syslog API. This isn't rocket science, it's far from being brittle, and it's well documented. In fact, it's one of the reasons that output redirection even exists. A simple command that could be tossed into a single shell script would be:

   ( my-application 2>&1 | my-syslog-shunt ) &

4. if you have the ability to alter your application's source code, you should write a shunt into it to dump the text output to syslog instead of a plain text file. This shouldn't be too hard; all you do is take the lines you would output, and wrap them with a call. However....

5. you might not have access to the source code at all, so you can't do this. Which means something like #3 above would work fine.