Tail the "in the last hour written lines from a log file" is it possible?
I am trying to grep looking for a pattern in to a log file, but I need the last hour of the log file. A tail -n XX will not work. Does anybody know if this is possible? Some kind of tail the "in the last hour written lines from a log file"
If there is any command or procedure I appreciate that.
Thanks a lot
Let's say your log have the following structure:
219.369.42.449 - - [05/Mar/2020:11:05:17 +0200] "log line"
219.369.42.449 - - [05/Mar/2020:11:06:37 +0200] "log line"
219.369.42.449 - - [05/Mar/2020:12:01:14 +0200] "log line"
219.369.42.449 - - [05/Mar/2020:12:07:23 +0200] "log line"
We can get all lines from the first occurrence of 05/Mar/2020:11
to the end $
of the file by using sed
in the following way:
sed -n '/05\/Mar\/2020:11/,$p' "/path/to/file.log"
The option
-n
will suppress the normal output ofsed
, but the flagp
will print the matched part of the file.Note, if there isn't presented any record that mach to
05/Mar/2020:11
,sed
wont provide any output.
We can automate the above by the help of the commands date
and eval
:
COMMAND="sed -n '/$(LANG=C date --date='1 hour ago' "+%d\/%b\/%Y:%H")/,\$p'"
eval $COMMAND \"/path/to/file.log\"
- Using
sed
with double quote marks and variable within the expression doesn't provide the desired output in this case. - So we first constructing the command as string and convert it to a real command by
eval
. -
LANG=C
(LANG=en_us_88591
) stands in order to get the desired date format, because, for example, in my case the default value of this envvar isbg_BG.UTF-8
.
You can create a script, based on the two lines above - examples of such script:
apache2 : How to search a string from apache2 error logs in specific time range?
modsecurity-whitelist-rule-generator.bash
- that parse events within ModSecutity'smodsec_audit.log
by their unique-id, and then generates whitelist rules for ModSecutity.
There is no command or option to tail that will track changes in the past hour. You will have to grep the timestamps in the log or keep tail -f
running and just scroll back when you need to check something. This has the advantage of also allowing you to catch events that happened 61 minutes ago.
When you run your command every 5 minutes, also make a copy of the log file. Then you can diff from the 12th-last copy you made to get the current changes.
The approach I normally take is:
tail -f {log}
This will have tail show messages -as they are written- to {log} and it will only end doing so when you stop the command. So what is shown is real time and logically also always within the current hour. The buffer size of the terminal session will be to where you can scroll back in time.