SFTP file lock mechanism

SFTP protocol supports locking since version 5. See the specification.

You didn't specify, what SFTP server are you using. So I'm assuming the most widespread one, the OpenSSH. The OpenSSH supports SFTP version 3 only, so it does not support locking.

Anyway, even if your server supported file locking, most SFTP clients/libraries won't support SFTP version 5. Or even if they do, they won't support the locking feature. Note that the lock is explicit, the client has to request it.

There are some common workarounds for the problem:

• As suggested by @user1717259, you can have the client upload a "done" file, once an upload finishes. Make your automated system wait for the "done" file to appear.

• You can have a dedicated "upload" folder and have the client (atomically) move the uploaded file to a "done" folder. Make your automated system look to the "done" folder only.

• Have a file naming convention for files being uploaded (".filepart") and have the client (atomically) rename the file after an upload to its final name. Make your automated system ignore the ".filepart" files.

  See (my) article Locking files while uploading / Upload to temporary file name for example of implementing this approach.

  Also, some FTP servers have this functionality built-in. For example ProFTPD with its HiddenStores directive (courtesy of @fakedad).

• A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes have not changed for some time interval.

• You can also make use of the fact that some file formats have clear end-of-the-file marker (like XML or ZIP). So you know, when you download an incomplete file.

A typical way of solving this problem is to upload your real file, and then to upload an empty 'done.txt' file.

The automated system should wait for the appearance of the 'done' file before trying to read the real file.

A simple file locking mechanism for SFTP is to first upload a file to a directory (folder) where the read process isn't looking. You can "make" an alternate folder using the sftp> mkdir command. Upload the file to the alternate directory, instead of the ultimate destination directory. Once the SFTP> put command completes, then do a move like this: SFTP> move alternate_path/filename destination_path/filename. Since the SFTP "move" is just switching the file pointers, it is atomic, so it is an effective lock.