Unix/Linux simple log parser (since, until)

Take a look at this Python program I wrote to see if it comes close to what you're looking for or can be adapted to your needs.


Even if you find one I don't know if I would trust it. For example, since the timestamp isn't known ahead of time, the only what it could distinguish DD-MM-YYYY and MM-DD-YYYY would be to read ahead until either xx or zz with xx-zz-YYYY is greater than 12. I am sure there are other issues.

Writing your own would be much easy and more reliable I think as you can use your language's standard string to datetime library and specify the date format specifiers explicitly.

For example with Python:

    import re
    from datetime import datetime
    line_regex = re.compile(
        r'''
        \[
        (?P<day> \d{1,2} )
        /
        (?P<month> \w{3} )
        / 
        (?P<year> \d{4} )
        :
        (?P<hour> \d{2} )
        : 
        (?P<minute> \d{2} )
        : 
        (?P<second> \d{2} )
        \s
        (?P<timezone> -?\d{4} )
        \]
        ''', re.VERBOSE)


   new_entry['time'] = datetime.strptime(
        parsed_line['day'] +
        parsed_line['month'] +
        parsed_line['year'] +
        parsed_line['hour'] +
        parsed_line['minute'] +
        parsed_line['second']
        , "%d%b%Y%H%M%S"
   )

If you do want a module / library that will try to figure out the format they do exist, one option for Perl is Date::Parse.


The only solution I've ever found for this which was even halfway decent was syslog-ng logging to a database (at which point it's reduced to simple SQL queries) -- Assuming you're logging everything centrally this doesn't add too much additional pain.

(Insert obvious benefits & obvious caveats here)