Terminal paste unreliable
Solution 1:
Stdout in this case is line-buffered and is 1024 bytes which is causing the output truncation on the display. When you paste from the clipboard the non-printable New-Line (EOL) characters at the end of each line are simply counted as another character on the line as far as the buffer count is concerned. The reason the math doesn’t add up for where the truncation happens that @miken32 was trying to calculate, is because you are not counting the New-Line characters.
If you cat the test file using a –e option you can see the non printable EOL characters as $, including them in the math should add up.
$ cat -e test
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
1234567890,1234567890,1234567890,1234567890,1234567890,1234567890$
You can get around filling the line buffer with pasted bytes if you specify your own end-of-file (EOF) marker to the cat command. This way when the cat command encounters the EOL characters it will reset the line buffer at the beginning of each new line instead of just counting them as another byte in a continuous stream because the cat command is parsing each new line looking for the EOF string on a line by itself to know when to exit.
You can do something like this :
cat << EOF > test
The stdin redirect (<< EOF) tells cat to keep printing until it encounters input matching the specified end-of-file characters on a new line by themselves. This way stdout will then print every pasted printable character line by line, the drawback is that you get a “>" character in the output at the start of each new line.
$ cat << EOF > test
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> 1234567890,1234567890,1234567890,1234567890,1234567890,1234567890
> EOF
The end-of-file marker is typed literally as the characters E, O, F, after your paste and can be anything you like. Also the EOF characters, and the > characters are not included in the redirected output sent to the file. Typing the letters as EOF is just symbolic and can be anything you like, XXX for example, you just have to be certain that what ever EOF marker you specify does not appear in the paste buffer.
For the record, you can still use ^D as the EOF marker when using stdin, even though you specify something else. The use of the EOF string is an old school scripting convention for delimiting a block of text to redirect from within the script.
Hope this helps.