Rsync statistics number of files
I'm using rsync with -vrlHh --delete --stats --force options to mirror two directories.
The first directory is the source and it's my external hd, the destination directory is empty because I've just created it.
I run rsync -vrlHh --delete --stats --force my_hd dest_dir and I get this output.
...
2012/05/12 11:59:29 [18094] Number of files: 189315
2012/05/12 11:59:29 [18094] Number of files transferred: 178767
2012/05/12 11:59:29 [18094] Total file size: 241.57G bytes
2012/05/12 11:59:29 [18094] Total transferred file size: 241.57G bytes
2012/05/12 11:59:29 [18094] Literal data: 241.57G bytes
2012/05/12 11:59:29 [18094] Matched data: 0 bytes
2012/05/12 11:59:29 [18094] File list size: 4.08M
2012/05/12 11:59:29 [18094] File list generation time: 0.002 seconds
2012/05/12 11:59:29 [18094] File list transfer time: 0.000 seconds
2012/05/12 11:59:29 [18094] Total bytes sent: 241.61G
2012/05/12 11:59:29 [18094] Total bytes received: 3.44M
2012/05/12 11:59:29 [18094] sent 241.61G bytes received 3.44M bytes 30.67M bytes/sec
2012/05/12 11:59:29 [18094] total size is 241.57G speedup is 1.00
My question is why Number of files and Number of file transferred are different if the destination directory was empty?
I believe you are experiencing http://lists.samba.org/archive/rsync/2008-April/020692.html.
In short, rsync uses the word "file" in different ways depending on context. In your first "Number of files" count it counts everything. In your second "Number of files transferred", it does not count symbolic links and directories as files.
Example:
$ mkdir test
$ touch test/testfile
$ ln -s testfile test/testlink
$ ls -FR test
test:
testfile testlink@
$ rsync -vrlHh --stats test test2
sending incremental file list
created directory test2
test/
test/testfile
test/testlink -> testfile
Number of files: 3
Number of files transferred: 1
Total file size: 8 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 67
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 126
Total bytes received: 38
sent 126 bytes received 38 bytes 328.00 bytes/sec
total size is 8 speedup is 0.05
$ ls -FR test2
test2:
test/
test2/test:
testfile testlink@
From author 'Mike Bombich' in [email protected] :
For stats, rsync uses the word "file" inconsistently. When reporting the total "Number of files", it indicates a total number of filesystem objects which consists of regular files, directories, symlinks, specials, and devices. When reporting number of "files" transferred, it refers only to regular files.
So if there are any non-regular files in there (inc. directories) they won't be included in the count.