Forcing rsync to convert file names to lower case

You can change the case of the resulting filenames on the target server after the rsync. I wouldn't attempt to do this mid-transfer (in case you need to restart the copy). As for making the change on the linux side, you'd need to determine if there are any conflicts. You will also need to determine if you need the directory names' case to be changed. Will all names be unique? If so, an appropriate find script coupled with the tr or rename command could do the job...

# Examples - Don't run directly
`rename 'y/A-Z/a-z/' *` # would change case on files within a directory.

You can mount a case-insensitive file system. Look at this post.

Also, this page suggests creating a disk image of type FAT32 and mounting it. The created fs will be case-insensitive such any Windows partition.

Using such a solution will eliminate the need to convert all these millions of files to lower-case.

Not the most elegant solution, but you can use LD_PRELOAD to override the relevant system calls and force everything to lower case. I thought it is fun so I did a little proof of concept and...

> ls in out
in:
CyltApJik  keumyomDu  LidusIcweo  spydjiPa  SycsEyror  tusUngEg

out:
> rsync -av in/ --rsync-path='env LD_PRELOAD=$PWD/lowercase.so rsync' localhost:out/ 
sending incremental file list
./
CyltApJik
LidusIcweo
SycsEyror
keumyomDu
spydjiPa
tusUngEg

sent 372 bytes  received 129 bytes  1002.00 bytes/sec
total size is 0  speedup is 0.00

> ls out
cyltapjik  keumyomdu  lidusicweo  spydjipa  sycseyror  tusungeg

And here is the sample, which may take a few iterations to become good enough to sync the whole thing.

> cat lowercase.c 
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#define __USE_GNU
#include <dlfcn.h>

static int (*real_lstat) (const char *, struct stat *) = NULL;
static int (*real_rename)(const char *, const char *)  = NULL;

char * lowered(const char * string)
{
        char * low = strndup(string, 2048);
        char * c;
        if (low == NULL) return NULL;
        for (c = low; *c; c++) {
                *c = tolower(*c);
        }
        return low;
}

int lstat(const char * path, struct stat * buf)
{
        int ret = 0;
        if (real_lstat == NULL) {
                real_lstat = dlsym(RTLD_NEXT, "lstat");
        }
        ret = real_lstat(path, buf);
        if (ret == 0) return ret;
        ret = real_lstat(lowered(path), buf);
        return ret;
}

int rename (__const char *__old, __const char *__new)
{
        if (real_rename == NULL) {
                real_rename = dlsym(RTLD_NEXT, "rename");
        }
        return real_rename(__old, lowered(__new));
}
> gcc -ldl -fPIC -shared -o lowercase.so lowercase.c

Forcing rsync to convert file names to lower case

Related

Recent Posts