Writing "tail -f" output to another file

Buffering is the problem.

Do it this way,

tail -f log.txt | egrep --line-buffered 'WARN|ERROR' | tee filtered_output.txt
#                       ^^^^^^^^^^^^^^^

Confirmed to work on Cygwin too.


It's probably a buffering issue. See this SO post on disabling the auto-buffering when using pipes. You can use the unbuffer command from expect:

$ unbuffer tail -f log.txt | egrep 'WARN|ERROR' | tee filtered_output.txt

Edit: Since you have a longer pipeline, you probably need to unbuffer each command (except the last):

$ unbuffer tail -f log.txt | unbuffer egrep 'WARN|ERROR' | tee filtered_output.txt

Edit 2: unbuffer is available on Cygwin from the expect source package (eg expect-20030128-1-src.tar.bz2, found in the expect/examples folder), but it's a very short script. If you have the expect package already installed, simply put this into a script called unbuffer in your /usr/local/bin directory:

#!/usr/bin/expect --
# Description: unbuffer stdout of a program
# Author: Don Libes, NIST

eval spawn -noecho $argv
set timeout -1
expect

On Debian, the unbuffer command is provided in the expect-dev package and is installed as expect_unbuffer.