How can I prevent vanishing error when using rsync with an nfs mount?

I'm in the process of moving files from an Ubuntu server to a Snow Leopard server. The Ubuntu server has an NFS share of around 6TB which I'd like to clone to the Snow Leopard server.

I mounted the nfs share on the Snow Leopard server and then ran

rsync -av /Volumes/FromUbuntu /Volumes/LocalCopy

After it copies around 100GB it complains that files have vanished. I'm assuming that for some reason the NFS links are going stale? What could be causing this? The Ubuntu server is not crashing and there are not connectivity issues that I am aware of. I wouldn't mind running to the rsync command over and over until the copy was finished, but it takes about 6 hours just for rsync to make it's file list.

Any suggestions? Would it be faster to just rsync over SSH? Thanks!

(PS: I've tried just using 'cp -arv' which doesn't seem to fail but according to the network traffic monitor on the Mac it seems to take twice as long to copy files as rsync after rsync has built its file list?)

UPDATE: I'm trying rsync over ssh from the mac server to the ubuntu server and it seems to been going much faster (it took less than two hours to create a file list and start transferring, where as when rsyncing from the nfs mount took around six hours just to build the initial file list.) It definitely looks like there is some problem with mounting the NFS share on the mac. Has anyone had problems with this? What about the other way around, mounting nfs shares on the mac on an ubuntu client? I was planning on hosting an NFS share for Ubuntu clients on the Mac, but now I'm getting nervous. Thanks for your input!

Solution 1:

As far as I know, the "file-has-vanished" condition is not a fatal error condition for rsync, and the process should continue just fine. Are you saying that rsync is halting upon reporting the "vanished file" messages?

If you are backing up a live filesystem, it is always possible to get the "vanished" message from rsync. Between the time that rsync adds a file to its transfer list and the time that rsync actually gets around to reading it, a file can "vanish" for any number of legitimate reasons (e.g. a temporary lock file was removed, or a file was renamed). Note that some applications work around limitations in NFS file locking by creating and removing hidden lock files, and it's possible that rsync may be seeing some of those.

I would suggest to keep your source filesystem as quiescent as possible while it is being backed up. Ideally, no one other than the backup process should be accessing it, but I realize that that may not be feasible. If your source filesystem is on an LVM logical volume, then you could consider creating a read-only snapshot to use as your backup source.

Solution 2:

This is a problem with special characters in the file names, as discussed here: https://superuser.com/questions/91967/rsync-character-set-problems

Solution 3:

What I do in my daily backup script, this script is a rsync wrapper :

#!/bin/bash

rsync "$@" 2> >(grep -v vanished)
ret=$?
((ret==24)) && exit 0 || exit $ret