Recursive directory download with Paramiko?

I want to download a directory with unknown contents recursively via SSH and have been trying Paramiko. I have seen several examples how to upload directories but none that covers recursive download.

I can list all items in a directory but haven't been able to find a way of knowing if the item is a file (to download) or a directory (to call recursively).

transport = paramiko.Transport((MY_IP, 22))
transport.connect(username=MY_NAME, password=MY_PASS)
sftp = paramiko.SFTPClient.from_transport(transport)

file_list = sftp.listdir(path='/home/MY_HOME_DIR')
    for item in file_list:
        # Here is an item name... but is it a file or directory?
        print(item)
sftp.close()
transport.close()

So how do I know if an item is a file or if it is a directory?

from stat import S_ISDIR

def isdir(path):
  try:
    return S_ISDIR(sftp.stat(path).st_mode)
  except IOError:
    #Path does not exist, so by definition not a directory
    return False

...assuming sftp is an open Paramiko SFTP connection.

An old question, but a solution I came up with that works quite well, it's a little bit sloppy (typecasting and slashes and all) - but it does work.

Note this uses fabric.api.local to make the directories in the destination.

def sftp_get_recursive(path, dest, sftp=sftp):
    item_list = sftp.listdir(path)
    dest = str(dest)

    if not os.path.isdir(dest):
        local("mkdir %s" % dest)

    for item in item_list:
        item = str(item)

        if is_directory(path + "/" + item, sftp):
            sftp_get_recursive(path + "/" + item, dest + "/" + item, sftp)
        else:
            sftp.get(path + "/" + item, dest + "/" + item)

A small update to Dan LaManna's answer that works in 2021.

import paramiko
import os
from stat import S_ISDIR, S_ISREG    

def sftp_get_recursive(path, dest, sftp):
    item_list = sftp.listdir_attr(path)
    dest = str(dest)
    if not os.path.isdir(dest):
        os.makedirs(dest, exist_ok=True)
    for item in item_list:
        mode = item.st_mode
        if S_ISDIR(mode):
            sftp_get_recursive(path + "/" + item.filename, dest + "/" + item.filename, sftp)
        else:
            sftp.get(path + "/" + item.filename, dest + "/" + item.filename)

transport = paramiko.Transport((host, port))
transport.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(transport)
sftp_get_recursive(remote_path, local_path, sftp)
sftp.close()

Paramiko does not support recursive operations.

But it's easy to implement:

import os
from stat import S_ISDIR, S_ISREG

def get_r_portable(sftp, remotedir, localdir):
    for entry in sftp.listdir_attr(remotedir):
        remotepath = remotedir + "/" + entry.filename
        localpath = os.path.join(localdir, entry.filename)
        mode = entry.st_mode
        if S_ISDIR(mode):
            try:
                os.mkdir(localpath)
            except OSError:     
                pass
            get_r_portable(sftp, remotepath, localpath)
        elif S_ISREG(mode):
            sftp.get(remotepath, localpath)

You also can use pysftp. It's a wrapper around Paramiko that has more Python-ish look and feel and supports recursive operations. See

pysftp.Connection.put_r()
pysftp.Connection.get_r()

Or see my answer to Python pysftp get_r from Linux works fine on Linux but not on Windows.

But pysftp seems to be an abandoned project, so you better stick with Paramiko.

You can use the stat() method of your sftp object:

http://www.lag.net/paramiko/docs/paramiko.SFTPClient-class.html

Recursive directory download with Paramiko?

Related

Recent Posts