Why is file transfer speed decreased when copying multiple smaller files?

Let's say I'm copying a portable apps directory from PC to an external HDD. An application.exe file is around 4 megabytes in size, but most other files are only tens of kilobytes in size, and most of those files take longer to copy/move than that bigger file having 4 megabytes.

Why is this?


If you transfer a file, theres an open+read+close at the source, and an open+write+close at the destination, with all the associated attributes (ctime/atime/mtime etc) being created at the destination. Fragmentation (higher seek times) will also affect the datarate. Congestion of e.g. a sata-channel may also be an issue (use a separate ports for each disk, buy a sata card if you run out). Bigger files have longer consecutive reads or writes, and will transfer faster. Im sure there's much more to it. Does this help you?


When transferring data, if you move one large file, you only have to write the directory entry once and then stream the data and write more or less continuously. When you have many files, you write the directory info, move to the file area and write the data, then go back and write the directory entry for the next file and then move back to the data area and write the data and back and forth. The overhead of moving back and forth adds up.


That's the way windows works, and that's the part HDD are bad at. Usually SSDs are good at copying small files. You can check out benchmarking of HDDs and SSDs about file transfer speeds. But you have to know that other things are in place, like windows is reading first the files, then your antivirus checks them, and then they get written to the new destination folder. Not to mention that when you cut a file, the deletion process is added. So, more files, more "real" time added to copy a bigger number of files.