What does the message, "too many links" mean when using CP on Linux?

I am copying 25,000 folders containing hundreds of files each from one hard drive to another on the same Linux server. The destination folder already contains 25,000 different folders full of their own files. I am using the following command:

cp -v -r /folder_to_copy_from/* /folder_to_copy_to/

The process completed about 20%, then all of a sudden, every time it tried to copy a folder it gave the message "too many links" and skipped all the files in that folder, moving on to the next folder. That folder was never created. CP printed this message for all the remaining folders.

I estimate there were ~30,000 folders in the folder_to_copy_to when it stopped working. I am using a EXT3 formatted drive with plenty of space to hold the files on Linux CentOS.

If there is a max number of subdirectories allowed in a directory, is there a way to increase it? Should I investigate different formatting standards? Any other thoughts on what is going on?

Solution 1:

There's a limit of about 32,000 links per inode. (smells like an unsigned short) Each folder makes a link to its parent folder, so you were going to try to make some 50k links to a single inode, which wasn't going to fly.

Perhaps you should reconsider your directory structure.