Randomly shuffling lines in Linux / Bash

Solution 1:

You should use shuf command =)

cat file1 file2 | shuf

Or with Perl :

cat file1 file2 | perl -MList::Util=shuffle -wne 'print shuffle <>;'

Sort: (similar lines will be put together)

cat file1 file2 | sort -R

Shuf:

cat file1 file2 | shuf

Perl:

cat file1 file2 | perl -MList::Util=shuffle -e 'print shuffle<STDIN>'

BASH:

cat file1 file2 | while IFS= read -r line
do
    printf "%06d %s\n" $RANDOM "$line"
done | sort -n | cut -c8-

Awk:

cat file1 file2 | awk 'BEGIN{srand()}{printf "%06d %s\n", rand()*1000000, $0;}' | sort -n | cut -c8-

Just a note to OS X users who use MacPorts: the shuf command is part of coreutils and is installed under name gshuf.

$ sudo port install coreutils
$ gshuf example.txt # or cat example.txt | gshuf

Here's a one-liner that doesn't rely on shuf or sort -R, which I didn't have on my mac:

while read line; do echo $RANDOM $line; done < my_file | sort -n | cut -f2- -d' '

This iterates over all the lines in my_file and reprints them in a randomized order.