tail -F still stops on logrotate sometimes

Linux ip-10-10-64-122 4.1.7-15.23.amzn1.x86_64 #1 SMP Mon Sep 14
23:20:33 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I'm using tail -F in a bash script which in turn loops over the results. This script is supposed to run indefinitely

tail -F /tmp/my.log |
grep --line-buffered "xyz" |
while read i
do
    echo "$i"
done
echo "end of read"

however while sometimes it works through a logrotate, other times it does not:

tail: '/tmp/my.log' has become inaccessible: No such file or directory

In this case I do not see "end of read" meaning the tail/while is still running/stuck. I can confirm the log file does get put back in place after logrotate within milliseconds as you would expect.

Logrotate config:

/tmp/mosquitto.log {
            rotate 500
            compress
            maxsize 40M
            nocreate
            missingok
            postrotate
                    /usr/bin/killall -HUP mosquitto
            endscript
    }

How can I force this to continue to tail?


As i suspected, your log rotation basically renames the current log file, and then creates a new one. You need to use the copytruncate logrotate option, which keeps the original log file, along with all of its properties. This will allow tail to follow it properly. tail cannot follow a file, that does not exist anymore (the case when copytruncate is not used).

copytruncate: Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one. It can be used when some program cannot be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log file stays in place.