apache2 : How to search a string from apache2 error logs in specific time range?

I've created a simple script, based on the suggested answer. The script has five input variables:

  • $1 - date from (string from)
  • $2 - date to (string to)
  • $3 - log file, full path and name
  • $4 - the first string to be searched
  • $5 - the second string to be searched

The content of the script is:

#!/bin/bash

# Escape all special characters: "[", "]", ":", " ", "."
s1="$(echo $1 | sed -e 's/\[/\\\[/g' -e 's/\]/\\\]/g' -e 's/\:/\\\:/g' -e 's/\ /\\\ /g' -e 's/\./\\\./g')"
s2="$(echo $2 | sed -e 's/\[/\\\[/g' -e 's/\]/\\\]/g' -e 's/\:/\\\:/g' -e 's/\ /\\\ /g' -e 's/\./\\\./g')"
s3="$3"
s4="$(echo $4 | sed -e 's/\[/\\\[/g' -e 's/\]/\\\]/g' -e 's/\:/\\\:/g' -e 's/\ /\\\ /g' -e 's/\./\\\./g')"
s5="$(echo $5 | sed -e 's/\[/\\\[/g' -e 's/\]/\\\]/g' -e 's/\:/\\\:/g' -e 's/\ /\\\ /g' -e 's/\./\\\./g')"
[ ! -z "$s5" ] && s5=".*$s5"

# Crop the log file
sudo sed -n "/$s1/,/$s2/p" "$s3" | grep --color=always "$s4$s5"

Let's call the script crop-log and place it in /usr/local/bin, thus it will be available as shell command system wide:

sudo touch /usr/local/bin/crop-log
sudo chmod +x /usr/local/bin/crop-log
sudo nano /usr/local/bin/crop-log
  • Copy the above script's content and use in nano: Shift+Insert for paste; Ctrl+O and Enter for save; Ctrl+X for exit.

Example of usage:

$ crop-log '[Tue Oct 03 07:35:08.000989 2017]' '04 07:35:07.663281' "/var/log/apache2/error.log"

[Tue Oct 03 07:35:08.000989 2017] [mpm_prefork:notice] [pid 1622] AH00163: Apache/2.4.18 (Ubuntu) mod_python/3.3.1 Python/2.7.12 OpenSSL/1.0.2g mod_perl/2.0.9 Perl/v5.22.1 configured -- resuming normal operations
[Tue Oct 03 07:35:08.001011 2017] [core:notice] [pid 1622] AH00094: Command line: '/usr/sbin/apache2'
[Wed Oct 04 07:35:07.559898 2017] [mpm_prefork:notice] [pid 1622] AH00171: Graceful restart requested, doing restart    AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message'
[Wed Oct 04 07:35:07.663176 2017] [:error] [pid 1622] python_init: Python version mismatch, expected '2.7.6', found '2.7.12'.
[Wed Oct 04 07:35:07.663275 2017] [:error] [pid 1622] python_init: Python executable found '/usr/bin/python'.
[Wed Oct 04 07:35:07.663281 2017] [:error] [pid 1622] python_init: Python path being used '/usr/lib/python2.7/:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'.
$ crop-log 'Oct 03 07:35:08' '04 07:35:07.663281' "/var/log/apache2/error.log" "[mpm_prefork:notice]"

[Tue Oct 03 07:35:08.000989 2017] [mpm_prefork:notice] [pid 1622] AH00163: Apache/2.4.18 (Ubuntu) mod_python/3.3.1 Python/2.7.12 OpenSSL/1.0.2g mod_perl/2.0.9 Perl/v5.22.1 configured -- resuming normal operations
[Wed Oct 04 07:35:07.559898 2017] [mpm_prefork:notice] [pid 1622] AH00171: Graceful restart requested, doing restart
$ crop-log '03 07:35:08' 'Oct 04 07:35:07.663281' "/var/log/apache2/error.log" "[mpm_prefork:notice]" "AH00171:"

[Wed Oct 04 07:35:07.559898 2017] [mpm_prefork:notice] [pid 1622] AH00171: Graceful restart requested, doing restart
$ crop-log '-H--' '-Z--' "/var/log/apache2/modsec_audit.log" '[id \"'

Message: Access denied with redirection to https://www.youtube.com/watch?v=gLmcGkvJ-e0 using status 302 (phase 2). Match of "ipMatchFromFile /web-security/modsecurity-ip-white.list" against "REMOTE_ADDR" required. [file "/etc/apache2/mods-enabled/security2.conf"] [line "73"] [id "150"]
Message: Access denied with redirection to https://www.youtube.com/watch?v=nb2evY0kmpQ using status 302 (phase 2). Match of "ipMatchFromFile /web-security/modsecurity-ip-white.list" against "REMOTE_ADDR" required. [file "/etc/apache2/mods-enabled/security2.conf"] [line "73"] [id "150"]
Message: Access denied with redirection to https://www.youtube.com/watch?v=z9Uz1icjwrM using status 302 (phase 2). Match of "ipMatchFromFile /web-security/modsecurity-ip-white.list" against "REMOTE_ADDR" required. [file "/etc/apache2/mods-enabled/security2.conf"] [line "73"] [id "150"]