lsync, unison or some other inotify auto-syncing tool..?
I critically need the sync to be immediate as the user needs to display the thumbnail straight away on a web page
So you've no control over server affinity in your load balancing? Time to get a new load balancer.
I looked at unison and lsync, but there doesn't seem to be a rpm available from the centOS5.5 repo for either.
Previously I did this by building replication into the application - after all, it's the first thing to know when new content needs to be replicated - it was just a few lines of PHP to publish a notice to the other nodes that content needed fetching - when a node has been offline for a while, I ran rsync before bringing it completely back online (this approach makes handling offline nodes much simpler and non-intrusive). However if I were to implement such a system today then I do it using overlay filesystems - perhaps an NFS at the bottom of the stack and copy up on access.
See also this post
HTH
C.