Fastest method of copying files
If we have a successful build on our build server (CCNET) all ASP.NET website files are copied to the virtual directory (%output_dir%) so non-developers can see/test the latest version of the website. At the end of the build the following bat file is executed.
rmdir /s /q "%output_dir%"
mkdir "%output_dir%"
xcopy "%source_dir%*" "%output_dir%" /e /c /i /q /-y
The problem is I find the copying slow and was wondering if there are any copy commands available in Windows 2008 that are faster than xcopy? The source and destination are on the same drive. Below are the arguments we use when copying.
/e = copies directories and sub directories including empty ones.
/c = continues copying even if there are errors
/i = if destination does not exist destination is directory
/q = don't display filenames
/-y = confirm overwrite
From a performance standpoint only, xcopy or robocopy will give you similar results. I ran through a couple of tests on a Windows Vista 64-bit SP2 box to do some comparisons. All copies were performed between a internal 7200 RPM Sata II disk and an external USB 2.0 drive or on the same internal drive itself where indicated. No special setup was done (make up your own mind if that invalidates/validates the test), only to input the command into a batch file to execute. PowerShell was used to capture the start and stop times. After a couple of passes here are the averages from the tools I played with:
File: 732,909,568 bytes (698 MB), 1 ISO file copied to different directory on the same internal disk.
copy 6 secs (ex. copy G:\folder1\* G:\folder2\)
xcopy 6 secs (ex. xcopy G:\folder1 G:\folder2 /I /E /Y /R)
robocopy 6 secs (ex. robocopy G:\folder1\ G:\folder2 /E /NP)
teracopy 28 secs (ex. TeraCopy.exe Copy G:\folder1\ G:\folder2\)
fastcopy 19 secs (ex. fastcopy.exe /auto_close G:\folder1 /to=G:\folder2)
File: 732,909,568 bytes (698 MB), 1 ISO file copied to external USB disk.
copy 36 secs (ex. copy G:\folder1\* I:\folder2\)
xcopy 35 secs (ex. xcopy G:\folder1 I:\folder2 /I /E /Y /R)
robocopy 36 secs (ex. robocopy G:\folder1\ I:\folder2 /E /NP)
teracopy 36 secs (ex. TeraCopy.exe Copy G:\folder1\ I:\folder2\)
fastcopy 38 secs (ex. fastcopy.exe /auto_close G:\folder1 /to=I:\folder2)
Files: 45,039,616bytes (42.9MB) 5 random files copied to external usb disk
copy 6 secs (ex. copy G:\folder1\* I:\folder2\)
xcopy 5 secs (ex. xcopy G:\folder1 I:\folder2 /I /E /Y /R)
robocopy 6 secs (ex. robocopy G:\folder1\ I:\folder2 /E /NP)
teracopy 12 secs (ex. TeraCopy.exe Copy G:\folder1\ I:\folder2\)
fastcopy 6 secs (ex. fastcopy.exe /auto_close G:\folder1 /to=I:\folder2)
Files/directoies: 1,087,180,800 bytes (1.01 GB), 27 files/8 directories copied to external USB disk.
copy *Not included in test
xcopy 57 secs (ex. xcopy G:\folder1 I:\folder2 /I /E /Y /R)
robocopy 58 secs (ex. robocopy G:\folder1\ I:\folder2 /E /NP)
teracopy 56 secs (ex. TeraCopy.exe Copy G:\folder1\ I:\folder2\)
fastcopy 60 secs (ex. fastcopy.exe /auto_close G:\folder1 /to=I:\folder2)
This is by no means an exhaustive test, but just throwing a quick real world scenario at some of the more popular tools in this genre shows that your pretty safe sticking with either xcopy or Robocopy (from a performance standpoint only). Also the Robocopy option /NP
(No Progress) saves you 0 time. That doesn't mean you cannot benefit from using something other than xcopy however. Robocopy is a great example (from Wikipedia):
Robocopy is notable for capabilities above and beyond the built-in Windows copy and
xcopy commands, including the following:
- Ability to tolerate network outages and resume copying where it previously left off (incomplete files are noted with a date stamp corresponding to 1980-01-01 and contain a recovery record so Robocopy knows from where to continue).
- Ability to correctly copy attributes, owner information, alternate data streams, auditing information, and timestamps by default, without the need for numerous often forgotten command line switches.
- Ability to correctly copy NTFS ACLs, (when /COPYALL provided), and to assert the Windows NT "backup right" (/B) so an administrator may copy an entire directory, including files denied readability to the administrator.
- Persistence by default, with a programmable number of automatic retries if a file cannot be opened.
- A "mirror" mode, which keeps trees in sync by optionally deleting files out of the destination that are no longer present in the source.
- Ability to copy large numbers of files that would otherwise crash the built-in XCOPY utility.
- A progress indicator on the command line that updates continuously.
- Ability to copy long file and folder names exceeding 256 characters — up to a theoretical 32,000 characters — without errors.
I typically use:
robocopy source dest /E /MIR
or some other variant of the parameters - perhaps /NFL /NDL /NS /NC /NP
to match the "quiet"
Actually, by eliminating the Network, you really restricted your testing. You might wish to consider using a network share, which is going to be bulk of admin work.
Additionally, you should use FTP, and skip CIFS altogether. Eseutil.exe is another utility you could throw in there. ( an Exchange util that can be used elsewhere, along with it's four dependent dlls. )
Then I'd like to see your results.
I use XCopy for the same purpose. What I also did was added another NIC to that server and put it on a separate subnet. Then made a direct connection to the server I am transferring files from. That way the 2 machines are moving the copied data via one subnet and users can still access via the second nic which is connected directly to the LAN.