How can I repeat the content of a file n times?

Solution 1:

You don't need input-duplicated.txt.

Try:

mycommand <(perl -0777pe '$_=$_ x 1000' input-data.txt)

Explanation

• 0777 : -0 sets sets the input record separator (perl special variable $/ which is a newline by default). Setting this to a value greater than 0400 will cause Perl to slurp the entire input file into memory.
• pe : the -p means "print each input line after applying the script given by -e to it".
• $_=$_ x 1000 : $_ is the current input line. Since we're reading the entire file at once because of -0700, this means the entire file. The x 1000 will result in 1000 copies of the entire file being printed.

Solution 2:

I was originally thinking that I would have to generate a secondary file but I could just loop the original file in Bash and use some redirection to make it appear as a file.

There are probably a dozen different ways of doing the loop but here are four:

mycommand <( seq 1000 | xargs -i -- cat input-data.txt )
mycommand <( for _ in {1..1000}; do cat input-data.txt; done )
mycommand <((for _ in {1..1000}; do echo input-data.txt; done) | xargs cat )
mycommand <(awk '{for(i=0; i<1000; i++)print}' input-data.txt)  #*

The third method there is improvised from maru's comment below and builds a big list of input filenames for cat. xargs will split this into as many arguments as the system will allow. It's much faster than n separate cats.

The awk way (inspired by terdon's answer) is probably the most optimised but it duplicates each line at a time. This may or may not suit a particular application, but it's lightning fast and efficient.

But this is generating on the fly. Bash outputting is likely to be very much more slow than something can read so you should generate a new file for testing. Thankfully that's only a very simple extension:

(for _ in {1..1000}; do echo input-data.txt; done) | xargs cat > input-duplicated.txt
mycommand input-duplicated.txt

Solution 3:

I would just use a text editor.

vi input-data.txt
gg (move cursor to the beginning of the file)
yG (yank til the end of the file)
G (move the cursor to the last line of the file)
999p (paste the yanked text 999 times)
:wq (save the file and exit)

If you absolutely need to do it via the command-line (this requires you to have vim installed, as vi doesn't have the :normal command), you could use:

vim -es -u NONE "+normal ggyGG999p" +wq input-data.txt

Here, -es (or -e -s) makes vim operate silently, so it shouldn't take over your terminal window, and -u NONE stops it from looking at your vimrc, which should make it run a little faster than it otherwise would (maybe much faster, if you use a lot of vim plugins).