Rsync stalls while transferring some files

My computer is running macOS 11 with rsync version 2.6.9 protocol version 29. My backup destination is a Synology NAS running DSM 7.0 with rsync version 3.1.2 protocol version 31. I access the Synology over SSH on local network. The NAS storage has a Btrfs filesystem.

For thousands of files the rsync process and the file transfers are correct, fast and as-expected. Then the process will stall in the middle of a file. When I kill and restart the rsync process it will stall at the same chunk of data in the same file. When I delete the partially transferred file in my backup destination and re-run the process, it will transfer the exact same amount of data and stall again. When I remove that file from my source dataset, and restart rsync it correctly adds thousands of files to the backup location, up to the point that it gets stalled on a different file on a specific chunk of data. And repeat.

I am running the following parameters for rsync, but because the progress just stalls rather than fails there is no useful output to explain the issue.

rsync -av --itemize-changes --progress --stats --human-readable /src user@host:/dest

The same data is backed up on a HFS+ formatted USB external hard drive, and the backup with rsync has no problem for that destination. So that would imply it's not the reading of data that is failing. What could be specific about the data chunk that it fails to write on the Synology?

Output while stalling the first time the file is attempted to be created:

<f+++++++ photos_and_videos/videos/1163537_1151511378282944_34972_n_10150522682652944.mp4
      54.23M  55%   17.11MB/s    0:00:02

Output while stalling every following attempt to run the process:

<f.stp... photos_and_videos/videos/1163537_1151511378282944_34972_n_10150522682652944.mp4
           0   0%    0.00kB/s    0:00:00

Solution 1:

The issue was caused by incorrect configuration of storage quota on the Synology (it was set too low). Rsync didn't fail on hitting the quota, likely because running out of space on the file system does not trigger the same result/error as hitting the storage quota on the Synology.