bash workflow for operating with big number of tar.gz archives
I am working with the directory consisted of 4 sub-dirrectories
ls -t
pnmrnp40_to_69 pnmrnp9028_to_9100 pnmrnp00_to_39 pnmrnp70_to_9028
inside of each prmnp* subdir there are many filles belonged either to *.tar.gz archive or *.md5sub (which I don not know what is it so it should be removed).
charlie@Precision-7920-Tower:~/Documents/script/mega_data/pnmrnp/pnmrnp40_to_69$ ls -t
ligands57_dir_results.tar.gz.md5sum ligands40_dir_results.tar.gz.md5sum
ligands57_dir_results.tar.gz ligands69_dir_results.tar.gz
ligands69_dir_results.tar.gz.md5sum ligands68_dir_results.tar.gz
ligands68_dir_results.tar.gz.md5sum ligands67_dir_results.tar.gz
ligands67_dir_results.tar.gz.md5sum ligands66_dir_results.tar.gz
ligands66_dir_results.tar.gz.md5sum ligands65_dir_results.tar.gz
I need a simple bash workflow which will move to each of the subdirectories
- remove all *.md5sub
- untar all *.tar.gz to the same subfolder (keeping the name of the original archive).
Here is my workflow in bash:
#!/bin/bash
# assuming that the script is in the folder contained all subdirectories
dir="$PWD"
# loop each subdirectory
for subdir in ${dir}
cd ${subdir}
# unzip each archive to the same place
for tar in *.tar.gz; do
tar xzvf $tar
done
# return to initial dir
cd ..
done
are there any possibilities to make this script more efficient in order that it could be adapted for very big number of archives?
find(1)
:
...
-execdir command ;
-execdir command {} +
Like -exec, but the specified command is run from the
subdirectory containing the matched file, which is not normally
the directory in which you started find. As with -exec, the {}
should be quoted if find is being invoked from a shell. This a
much more secure method for invoking commands, as it avoids race
conditions during resolution of the paths to the matched files.
As with the -exec action, the `+' form of -execdir will build a
command line to process more than one matched file, but any
given invocation of command will only list files that exist in
the same subdirectory. If you use this option, you must ensure
that your $PATH environment variable does not reference `.';
otherwise, an attacker can run any commands they like by leaving
an appropriately-named file in a directory in which you will run
-execdir. The same applies to having entries in $PATH which are
empty or which are not absolute directory names. If any
invocation with the `+' form returns a non-zero value as exit
status, then find returns a non-zero exit status. If find
encounters an error, this can sometimes cause an immediate exit,
so some pending commands may not be run at all. The result of
the action depends on whether the + or the ; variant is being
used; -execdir command {} + always returns true, while
-execdir command {} ; returns true only if command returns 0.
...
find -type f -name '*.tar.gz' -execdir tar xvf {} \;