How to copy directory recursively in python and overwrite all?

Solution 1:

You can use distutils.dir_util.copy_tree. It works just fine and you don't have to pass every argument, only src and dst are mandatory.

However in your case you can't use a similar tool likeshutil.copytree because it behaves differently: as the destination directory must not exist this function can't be used for overwriting its contents.

If you want to use the cp tool as suggested in the question comments beware that using the subprocess module is currently the recommended way for spawning new processes as you can see in the documentation of the os.system function.

Solution 2:

Have a look at the shutil package, especially rmtree and copytree. You can check if a file / path exists with os.paths.exists(<path>).

import shutil
import os

def copy_and_overwrite(from_path, to_path):
    if os.path.exists(to_path):
        shutil.rmtree(to_path)
    shutil.copytree(from_path, to_path)

Vincent was right about copytree not working, if dirs already exist. So distutils is the nicer version. Below is a fixed version of shutil.copytree. It's basically copied 1-1, except the first os.makedirs() put behind an if-else-construct:

import os
from shutil import *
def copytree(src, dst, symlinks=False, ignore=None):
    names = os.listdir(src)
    if ignore is not None:
        ignored_names = ignore(src, names)
    else:
        ignored_names = set()

    if not os.path.isdir(dst): # This one line does the trick
        os.makedirs(dst)
    errors = []
    for name in names:
        if name in ignored_names:
            continue
        srcname = os.path.join(src, name)
        dstname = os.path.join(dst, name)
        try:
            if symlinks and os.path.islink(srcname):
                linkto = os.readlink(srcname)
                os.symlink(linkto, dstname)
            elif os.path.isdir(srcname):
                copytree(srcname, dstname, symlinks, ignore)
            else:
                # Will raise a SpecialFileError for unsupported file types
                copy2(srcname, dstname)
        # catch the Error from the recursive copytree so that we can
        # continue with other files
        except Error, err:
            errors.extend(err.args[0])
        except EnvironmentError, why:
            errors.append((srcname, dstname, str(why)))
    try:
        copystat(src, dst)
    except OSError, why:
        if WindowsError is not None and isinstance(why, WindowsError):
            # Copying file access times may fail on Windows
            pass
        else:
            errors.extend((src, dst, str(why)))
    if errors:
        raise Error, errors

Solution 3:

Here's a simple solution to recursively overwrite a destination with a source, creating any necessary directories as it goes. This does not handle symlinks, but it would be a simple extension (see answer by @Michael above).

def recursive_overwrite(src, dest, ignore=None):
    if os.path.isdir(src):
        if not os.path.isdir(dest):
            os.makedirs(dest)
        files = os.listdir(src)
        if ignore is not None:
            ignored = ignore(src, files)
        else:
            ignored = set()
        for f in files:
            if f not in ignored:
                recursive_overwrite(os.path.join(src, f), 
                                    os.path.join(dest, f), 
                                    ignore)
    else:
        shutil.copyfile(src, dest)