rename all jpeg files sequentially in all nested subfolders and move to a new folder

My question is similar to this one

https://stackoverflow.com/questions/54811336/rename-jpeg-files-sequentially-in-multiple-subfolders

but my folder structure is like this

├── mainfolder
|    ├── subfolder1
|    |   ├── 01folder
             |--02folder
                 |--03folder 
                      |--- 0001.jpg
|    |                └── 0002.jpg
                      ...
|    |                └── 0250.jpg
|    |
|    ├── subfolder2
           |── 01folder
                |--02folder
                     |--03folder 
|    |                  ├── 0001.jpg
|    |                  └── 0002.jpg
                         ...
|    |                  └── 0250.jpg
|    |
|    ├── subfolder3
           ├── 01folder
                 |--02folder
                     |--03folder  
|    |                   ├── 0001.jpg
|    |                   └── 0002.jpg
                         ...
|    |                   └── 0250.jpg
…

all folders have jpg images numbered from 1 to 250 for a total of 100000 images. I would like to move all the files and rename them before moving them so the folder structure looks like this

├── newfolder
|    |--1.jpg
|    |--2.jpg
|    |--3.jpg
...
|    |--100000.jpg

I am currently using this

to move all the files from the nested subdirectories to a new directory

find ~/mainfolder/ -type f -print0 | xargs -0 mv -t ~/newfolder/

How do I move files out of nested subdirectories into another folder in ubuntu? (Trying to strip off many subfolders)

not sure how to rename them as well though as they will overwrite each other due to having the same name when moving resulting in only 250 files instead of 100000 files.

Solution 1:

With Larry Wall's rename/prename(*) you can easily "flatten" the structure:

cd mainfolder
rename -n 's:/:-:g' */*/*.jpg

This will move subfolder1/0001.jpg to subfolder1-0001.jpg (in mainfolder) because rename edits the whole input path and then does the equivalent of mv using source and edited names.

• g in the edit means to replace all occurrences, useful if you have more than one level of subdirectories.
• -n just shows what it would do without doing anything
• remove or replace by -v for the actual action.
• by default no overwrite can happen (you need a -f) but normally since this includes subdirectories in the names you shouldn't have name clashes anyway.
• once everything is in mainfolder, it is easy to do a bulk move elsewhere if needed.

• if the files are in various levels of nesting, use shopt -s globstar and use ** to denote all levels:

  rename s:/:-:g **/*.jpg
  

For instance:

Starting with

.
├── sub1
│   ├── subsub1
│   │   ├── 001.jpg
│   │   ├── 002.jpg
│   │   └── 003.jpg
│   └── subsub2
│       ├── 001.jpg
│       ├── 002.jpg
│       └── 003.jpg
└── sub2
    ├── subsub1
    │   ├── 001.jpg
    │   ├── 002.jpg
    │   └── 003.jpg
    └── subsub2
        ├── 001.jpg
        ├── 002.jpg
        └── 003.jpg

Execute:

rename s:/:-:g */*/*.jpg

This results in:

.
├── sub1
│   ├── subsub1
│   └── subsub2
├── sub1-subsub1-001.jpg
├── sub1-subsub1-002.jpg
├── sub1-subsub1-003.jpg
├── sub1-subsub2-001.jpg
├── sub1-subsub2-002.jpg
├── sub1-subsub2-003.jpg
├── sub2
│   ├── subsub1
│   └── subsub2
├── sub2-subsub1-001.jpg
├── sub2-subsub1-002.jpg
├── sub2-subsub1-003.jpg
├── sub2-subsub2-001.jpg
├── sub2-subsub2-002.jpg
└── sub2-subsub2-003.jpg

And you can get rid of the subdirectories using

rm -r sub*/

(final slash is important! it restricts the match to directories)

So finally:

.
├── sub1-subsub1-001.jpg
├── sub1-subsub1-002.jpg
├── sub1-subsub1-003.jpg
├── sub1-subsub2-001.jpg
├── sub1-subsub2-002.jpg
├── sub1-subsub2-003.jpg
├── sub2-subsub1-001.jpg
├── sub2-subsub1-002.jpg
├── sub2-subsub1-003.jpg
├── sub2-subsub2-001.jpg
├── sub2-subsub2-002.jpg
└── sub2-subsub2-003.jpg

If you have a huge collection of files, you cannot process in one call because the file expansion of **/*.jpg will generate a list which is too long (/bin/sh: 1: rename: Argument list too long`). So you have to split the work, for instance by calling rename once by directory:

shopt -s globstar 
for d in **/; do rename s:/:-:g "$d"*.jpg; done

(the / in '**/' is important, it restricts the expansion to directories)

If you still get the Argument list too long message then you have a directory with very many files :) You just have to find a way to split the work, for instance:

for f in path/to/hugedir/A*;jpg; do rename s:/:-:g "$d"*.jpg; done
for f in path/to/hugedir/B*;jpg; do rename s:/:-:g "$d"*.jpg; done
[... etc ...]

If this is still too big and or you want to be blunt, you can also call rename once for each file:

shopt -s globstar 
for f in **/*.jpg; do rename s:/:-:g "$f"; done

(this will work because here the expansion of "**/*.jpg" is just a variable in your shell, and not the list of arguments to a command).

PS: bash assumed here, if you are putting this in a script, make sure the "shebang" is /bin/bash, on some systems sh is a lightweight shell (dash) that doesn't support globstar.

(*) name varies depending on distro, rename for Debian/Ubuntu and derivatives, and IIRC prename on RedHat and derivatives.