How do `>` and `>>` work?

I tried to rev a file and then pipe it to cat > same_file but it was turning into a blank file.

While I tried rev file.txt | cat > file2.txt && mv file.txt file2.txt; it worked.

Even rev file.txt | cat >> file.txt; worked.

But when I tried rev file.txt | cat > file.txt it failed.

Solution 1:

The basic thing you need to grasp in this case between both ways of redirecting (> and >>) is:

>

Redirects and overwrites the information where it was pointed to. This happens while receiving any information through pipe "|"

>>

Redirects and concatenates to the information it was pointed to. This happens while receiving any information through pipe "|"

In both cases if the file does not exist, it will be created instead. Only on ">>" will the information be concatenated if you run it again on the same file. With ">" you would simply overwrite everything you did on the first run.

But here is the deal when using the same input file as the output file. In that particular case, if you use ">" you would be removing the information the "input" part needs to parse, since the output file would be "overwriting it". So in:

rev file.txt | cat > file.txt

What is actually happening in "slow motion explanation" is:

1. rev prepares to reverse the content of file.txt and send it to pipe
2. While rev is sending the information to pipe, pipe streams it directly to cat.
3. While cat is receiving the information it will automatically apply it to the file.txt it was set with.
4. The keyword here is "while", because everything is happening at the same time. Please see the excellent comments below by Emil to have a deeper understanding on this part.
5. cat will not wait for rev to pipe the whole file. It will simply start the minute the first part of the information gets to it, which means, depending on what symbol you used, it will open a connection to file.txt.
6. In this case since you used the > instead of >>, the shell will truncate the output file which means it will open and clear the information of file.txt while waiting for the new information to get to it. With >> it would open a connection with file.txt and wait for new information on the last line detected.
7. Since the information was already cleared in file.txt with >, rev would try to do it's work and get nothing because cat deleted everything in preparation for the new information.

So why do the others work after reading the above. Because of this:

rev file.txt | cat > file2.txt && mv file.txt file2.txt

Here you are piping to cat which is sending the information to another file. In this case, the processed input file file.txt is not the same as the output file file2.txt. After that you are literally overwriting the whole file2.txt with file.txt, so all the process made by cat was deleted. Basically the whole line could be simplified like cp file.txt file2.txt because it is doing the same thing since file2.txt at the end looses the rev and is overwritten with the mv command.

rev file.txt | cat >> file.txt

In this case you are concatenating the information to the same file. So it is only opening a connection to that file but not erasing the information as seen with a single >. The end result should be, the original information plus the reversed information.

Solution 2:

When the shell sees redirection, it opens the relevant files first, before executing any of the commands involved. Thus, when you do:

foo file.txt | bar > file.txt

The redirection to file.txt causes it to be truncated before foo is run and can read file.txt. On a side note, this is why you cannot do:

sed 'blah' file.txt > file.txt

And why sed has an in-place editing option.

Lastly, doing:

.. | cat > file.txt

is a useless use of cat, especially so if you are attempting to read from file.txt earlier on.

If you want to reverse a file in-place, there are no shortcuts. You might be able to use the sed or awk tricks with in-place editing.