MacOSX 10.6.7 cuts off stdin at 1024 chars

I've written a little perl script that I invoke as follows:

perl -pe'...' <a

I wanted to copy the contents of the input file 'a' from a web browser (a whole wordpress blog entry, to be exact). The copy part works perfectly, the full contents show up in ClipMenu. But when I try to paste it into a file using cat and redirect:

cat >a

It emits exactly 1024 characters, then stops accepting input - only CTRL-C can help. I've tried pasting only 1023 bytes/chars, and that works, after pressing enter, I could enter it into the file without any problems. But if I entered 1024 chars on a single line, there was no way to do anything else other than CTRL-C.

I've tried iTerm and Terminal.app, both show the exact same behavior. I've tried bash from macports and /bin, with the exact same results. I've even tried dd:

dd bs=1M of=a

To the exact same result. Nothing seems to help.

I'm clueless - why does this happen? And how could I copy-paste into a file on Mac OS X easily, and in a way that I can be sure the contents are not altered in any way?

% pbpaste | perl -e '...'

You're hitting the maximum line length for the tty. If you switch the tty out of line mode, it'll work.

But pbpaste is really what you want.

Every unix I've ever used has a limit on the length of cooked-mode input, though it probably differs from OS to OS. That's why you can only type so much in one line when you do a cat>a. The OS buffers input from the TTY so that you can do editing (e.g., backspace over text and change it) before sending it to the applciation (cat, in this case). The OS allocates a fixed-size buffer for this, and you're running into its limit. Your best option is to use an interactive program to write the file. For example, you could run vi and enter the

:set paste

command to turn off auto-indenting. Press "i" to enter insert mode, paste your code in, hit escape, and run:

:w a
:q

To save the file and quit.