Unable to rsync due to broken pipe

I'm trying to mirror a large Mongo database between from a production server to a dev environment by stopping Mongo on both servers and then running the command:

rsync --archive --delete --recursive --verbose --compress --rsh "ssh -t -o StrictHostKeyChecking=no -i key.pem" remoteuser@remotehost:/var/lib/mongodb/ /var/lib/mongodb

It runs fine for a few minutes, but then stopped with the error:

receiving incremental file list
./
collection-228--5129329295041693519.wt
inflate returned -3 (0 bytes)
rsync error: error in rsync protocol data stream (code 12) at token.c(557) [receiver=3.1.1]
rsync: [generator] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(820) [generator=3.1.1]

Googling the error suggests it's some sort of network connection problem, but I'm able to connect to both servers just fine.

If I re-run the command, it fails at the exact same file with the same error message. What's causing this error and how do I fix it?

Solution 1:

It turns out rsync accesses rsync on the remote server, and the versions where not the same between my servers. I was running 3.1.1 on the destination server, but 3.1.0 on the source server, and apparently this was enough to break the download of certain files. I install 3.1.1 on the source server, and afterwards the transfer worked flawlessly.

Solution 2:

Solution:

As explained by Cerin here, just upgrade your rsync versions to ensure that they are the exact same on both the sending and receiving PCs!

I had this exact same problem: a broken pipe write error when transferring a very large file over rsync's ssh, with rsync's --compress (-z) flag set.

My rsync error was:

rsync: [sender] write error: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at io.c(837) [sender=3.1.0]

rsync --version from my SENDING PC showed rsync version 3.1.0 protocol version 31, whereas from my RECEIVING PC it showed rsync version 3.1.2 protocol version 31. Therefore, I decided to just upgrade my SENDING PC to version 3.1.2 as well. Once I did that, it worked!

How to upgrade your `rsync` version:

Simply follow these instructions here to install rsync from source: http://www.beginninglinux.com/home/backup/compile-rsync-from-source-on-ubuntu.

In short:

Check your rsync version so you know what you currently have:
```
 rsync --version
```
Download your desired source file for the version you want here: https://download.samba.org/pub/rsync/src/.
In your GUI file manager, right-click and extract it.
Build from source, and install:
```
 ./configure
 make
 sudo checkinstall
```
Check your rsync version to ensure it was updated:
```
 rsync --version
```
Done!
(I haven't tested this, but apparently): To remove it from your system use:
```
 dpkg -r rsync
```

Once I did the upgrade to get both systems' rsync versions the same, it worked perfectly!

https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1300367
https://unix.stackexchange.com/questions/242898/rsync-keeps-disconnecting-broken-pipe/547861#547861
Google search for "rsync write error broken pipe" - https://www.google.com/search?q=rsync+write+error+broken+pipe&oq=rsync+write+error+&aqs=chrome.0.0j69i57j0l4.2263j0j7&sourceid=chrome&ie=UTF-8&sec_act=d
[my answer] https://askubuntu.com/questions/791002/how-to-prevent-sshfs-mount-freeze-after-changing-connection-after-suspend/942820#942820