SFTP file lock mechanism

by

SFTP protocol supports locking since version 5. See the specification.

You didn’t specify, what SFTP server are you using. So I’m assuming the most widespread one, the OpenSSH. The OpenSSH supports SFTP version 3 only, so it does not support locking.

Anyway, even if your server supported file locking, most SFTP clients/libraries won’t support SFTP version 5. Or even if they do, they won’t support the locking feature. Note that the lock is explicit, the client has to request it.

There are some common workarounds for the problem:

  • As suggested by @user1717259, you can have the client upload a “done” file, once an upload finishes. Make your automated system wait for the “done” file to appear.

  • You can have a dedicated “upload” folder and have the client (atomically) move the uploaded file to a “done” folder. Make your automated system look to the “done” folder only.

  • Have a file naming convention for files being uploaded (“.filepart”) and have the client (atomically) rename the file after an upload to its final name. Make your automated system ignore the “.filepart” files.

    See (my) article Locking files while uploading / Upload to temporary file name for example of implementing this approach.

    Also, some FTP servers have this functionality built-in. For example ProFTPD with its HiddenStores directive (courtesy of @fakedad).

  • A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes have not changed for some time interval.

  • You can also make use of the fact that some file formats have clear end-of-the-file marker (like XML or ZIP). So you know, when you download an incomplete file.

More Related Contents:

  1. Reference for proper handling of PID file on Unix
  2. Is there an advantage to use a Synchronized Method instead of a Synchronized Block?
  3. Volatile vs. Interlocked vs. lock
  4. Any way to select without causing locking in MySQL?
  5. Python sharing a lock between processes
  6. Python pysftp get_r from Linux works fine on Linux but not on Windows
  7. pysftp vs. Paramiko
  8. Screen orientation lock
  9. std::lock_guard or std::scoped_lock?
  10. How would you implement your own reader/writer lock in C++11?
  11. Does a locked object stay locked if an exception occurs inside it?
  12. Paramiko’s SSHClient with SFTP
  13. Command-line tool for finding out who is locking a file
  14. Turning on screen programmatically
  15. How to lock a single row
  16. PHP & Sessions: Is there any way to disable PHP session locking?
  17. What’s the least invasive way to read a locked file in C# (perhaps in unsafe mode)?
  18. x86 LOCK question on multi-core CPUs
  19. FTP Over SSH (SFTP) In delphi 2010
  20. Chunked encoding and content-length header
  21. file locking in php
  22. Google Cloud Platform solution for serverless log ingestion (files downloading) from a SFTP server
  23. scp or sftp copy multiple files with single command
  24. Transfer files to/from session I’m logged in with PuTTY
  25. Batch file for PuTTY/PSFTP file transfer automation
  26. Client-Server: File transfer from Android to PC connected via socket
  27. List files on SFTP server matching wildcard in Python using Paramiko
  28. Threads – Why a Lock has to be followed by try and finally
  29. EF Core takes a lot of time, sometimes, to perform SELECT query
  30. Are locks unnecessary in multi-threaded Python code because of the GIL?

Leave a Comment