Windows copy command concatenates several files in non-numerical order

I have a folder with around 5000 .ts files that I want to concat together into one. They follow the naming format of chunk_n so for example chunk_50, chunk_51, chunk_52 etc. When I run the windows copy command like this:

copy /b *.ts final.ts

It doesn't iterate through them in normal numeric order. It will start concatting them in this order:

chunk_100, chunk_1000, chunk_1001, chunk_1002, chunk_1003, chunk_1004, chunk_1005, chunk_1006, chunk_1007, chunk_1008, chunk_1009, chunk_101

and so on.

It doesn't look at the entire filename when it iterates over them it seems. Even if I remove the chunk_ beforehand and just named them all 1 through 5000, the issue would still persist. Is this intended behaviour? And is there a better way for me to concat all of these files?

In most programs, numeric order has never been normal order. Only recently did file managers (such as Windows Explorer) begin to sort files using "natural sort", but the copy command and the * have not been changed accordingly.

Instead, * expansion sorts filenames by their ASCII byte order, comparing each character separately. For example, with "chunk_1000.ts" vs "chunk_101.ts" (the extension is part of the name), the first 8 bytes (chunk_10) are equal, and then you have 0 < 1 (zero is ASCII 48, one is ASCII 49), and therefore chunk_100(0.ts) < chunk_101(.ts) – even if more digits follow, they're not considered to be part of a whole "number" at all.

(Byte order is the most straightforward way to sort names – all that's needed is directly comparing two memory locations. This simplicity was important when RAM was measured in kilobytes. Now it has been the case for a good 50 years, so it's pretty much impossible to change now, due to many scripts relying on the existing behavior.)

The simplest workaround is to pad all numbers to a fixed width, e.g. chunk0050 chunk0100.

A different workaround, available in more advanced shells such as PowerShell or Bash, is to get and sort the list of inputs as a separate step. For example, on Linux you would use the external 'sort' or 'natsort' commands to get the desired order (they're not exactly part of Bash, but generally present wherever Bash is present):

(printf '%s\n' chunk*.ts | sort -V | xargs -d'\n' cat) > allchunks.ts

In PowerShell, Sort-Object can be given a custom comparator; there isn't a pre-defined one for "natural" (numeric) order, but from here you can borrow a one-liner:

$ToNatural = { [regex]::Replace($_, '\d+', { $args[0].Value.PadLeft(20) }) }
$filenames = Get-ChildItem chunk*.ts | Sort-Object $ToNatural
$filenames = $filenames -join "+"
cmd /c "copy $filenames allchunks.ts"

File names are names, not numbers, so they're compared as text strings, using lexicographic order. chunk_1 comes before chunk_10 which comes before chunk_10a which comes before chunk_1z which comes before chunk_2.

If zsh is available, you can use the following command:

<*.ts(n) >final.ts

This leverages a combination of zsh features:

• Redirections without a command copy the input to the input.
• With multiple input redirections, the inputs are concatenated.
• The parentheses after the wildcard pattern contain glob qualifiers which control which files are matched and how they are sorted. The glob qualifier n applies numerical sorting of integer parts instead of the default lexicographic order, so chunk9.ts is sorted before chunk10.ts.

If basic Unix-like utilities are also available, you can use the cat command to concatenate files:

cat *.ts(n) >final.ts

Zsh is available through WSL (sudo apt-get install zsh), Cygwin and probably other distributions of free software for Windows.

The Windows copy command is a CMD.EXE builtin command. It does no sorting at all on its wildcard expansions. This behaviour dates back to at least MS-DOS 6.22 COMMAND.COM (I don't have anything earlier handy to test with). It just so happens that your NTFS file system stores file names in a B-tree structure which has the effect of enumerating them in an order that approximates lexical sorted order.

More information is available on the Old New Thing blog:

Why do NTFS and Explorer disagree on filename sorting?

What order does the DIR command arrange files if no sort order is specified?