How can I view a progress bar when running rsync?
rsync has a --info
option that can be used to not only output the current progress, but also the transfer rate and elapsed time:
--info=FLAGS fine-grained informational verbosity
The explanation of how to use it comes under the -P
option in the man page:
-P The -P option is equivalent to --partial --progress. Its purpose is to
make it much easier to specify these two options for a long transfer that
may be interrupted.
There is also a --info=progress2 option that outputs statistics based on
the whole transfer, rather than individual files. Use this flag
without out‐putting a filename (e.g. avoid -v or specify --info=name0)
if you want to see how the transfer is doing without scrolling the screen
with a lot of names. (You don’t need to specify the --progress
option in order to use --info=progress2.)
So the following:
rsync -r --info=progress2 --info=name0 "$src" "$dst"
Results in the following being output and continuously updated:
18,757,542,664 100% 65.70MB/s 0:04:32 (xfr#1389, to-chk=0/1510)
Note that when the transfer starts the total number of chunks, and therefore the current progress, can change when the recursive option is used as more files are discovered for syncing
You can use --progress
and --stats
parameters.
rsync -avzh --progress --stats root@server:/path/to/file output_name
root@server's password:
receiving incremental file list
file
98.19M 54% 8.99MB/s 0:00:08
How about this?
rsync_param="-av"
rsync "$rsync_param" a/ b |\
pv -lep -s $(rsync "$rsync_param"n a/ b | awk 'NF' | wc -l)
-
$rsync_param
Avoids double input of parameters
-
$(rsync "$rsync_param"n a/ b | awk 'NF' | wc -l)
Determines the number of steps to complete.
-
a/ b
-
a/
is the source -
b
is the target
-
Yeah, do what Jon said: use the --info=progress2
option. But, what do I do if my version of rsync is too old and doesn't support this option? Answer: upgrade rsync!
Here's how to build rsync
from source on Ubuntu
(tested on Ubuntu 16.04)
-
Download latest version of rsync: https://download.samba.org/pub/rsync/src/. Ex: "rsync-3.1.3.tar.gz". Save it in a directory WITH NO SPACES AT ALL to ensure it builds right.
-
In your folder explorer, right-click it and go to "Extract Here".
-
Enter the extracted folder (ex: "rsync-3.1.3")
-
Right-click the screen in your folder manager and go to "Open in Terminal." Alternatively, do steps 2 through 4 manually on the command line. Ultimately you just need to be
cd
ed into this extracted directory containing thersync
source code. -
Check current version of
rsync
. Make note of this so you can see later it actually got updated.rsync --version
-
Install necessary tools:
sudo apt update sudo apt install yodl
-
Build:
./configure make sudo make install
-
Ensure it was updated:
rsync --version
Sample output:
$ rsync --version
rsync version 3.1.3 protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, no ACLs, xattrs, iconv, symtimes, prealloc
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
-
Search the man pages for "progress2". You'll now have access to the
--info=progress2
option:man rsync
...then press /
key and type progress2
; press Enter to search for it; press n
for the 'n'ext match until you find the entry you're looking for:
There is also a
--info=progress2
option that outputs statistics based on the whole transfer, rather than individual files. Use this flag without outputting a filename (e.g. avoid-v
or specify--info=name0
) if you want to see how the transfer is doing without scrolling the screen with a lot of names. (You don’t need to specify the--progress
option in order to use--info=progress2
.)
Also see:
- [my answer] "How to use rsync": https://superuser.com/questions/1271882/convert-ntfs-partition-to-ext4-how-to-copy-the-data/1464264#1464264
Partial References:
- http://www.beginninglinux.com/home/backup/compile-rsync-from-source-on-ubuntu
This finally worked:
rsync "$rsync_param" -a --prune-empty-dirs --exclude "*.iso" rsync://archive.ubuntu.com/ubuntu/indices/ /repo/ubuntu/indices | pv -lep -s $(rsync "$rsync_param"n rsync://archive.ubuntu.com/indices/ /repo/ubuntu/indices | awk 'NF' | wc -l)