I have a directory of files with a structure like below:
./DIR01/2019-01-01/Log.txt ./DIR01/2019-01-01/Log.txt.1 ./DIR01/2019-01-02/Log.txt ./DIR01/2019-01-03/Log.txt ./DIR01/2019-01-03/Log.txt.1 ... ./DIR02/2019-01-01/Log.txt ./DIR02/2019-01-01/Log.txt.1 ... ./DIR03/2019-01-01/Log.txt ...and so on. Each DIRxx directory has a number of subdirectories named by date, which themselves have a number of log files that need to be concatenated. The number of text files to concatenate varies, but could theoretically could be as many as 5. I would like to see the following command performed for each set of files within the dated directories:
cd ./DIR01/2019-01-01/ cat Log.txt.4 Log.txt.3 Log.txt.2 Log.txt.1 Log.txt > ../../Log.txt_2019-01-01_DIR01.txt (I understand the above command will give an error that certain files do not exist, but the cat will do what I need of it anyways) Aside from cding into each directory and running the above cat command, how can I script this into a Bash shell script?
2 Answers
As usual for Linux, there are several possible ways of doing what you ask for.
For example, using bash's glob extension to run a script on all of the folders:
for dir in */* ; do # iterate through all 2nd-level subdirectories if [ -d "$dir" ] ; then # only run on folders cat $dir/Log.txt* > Log.txt_${dir////_} # that mess of slashes replaces them with underscores in the filename fi done Alternatively, you could use find:
for dir in $(find -mindepth 2 -maxdepth 2 -type d) cat $dir/Log.txt* > Log.txt_${dir////_} done Or probably any number of other solutions.
Thanks, but the solution I found on StackOverflow was to use the below code:
for dir in DIR*/*; do date=${dir##*/}; dirname=${dir%%/*}; cat $dir/Log.txt.{5..1} $dir/Log.txt > Log.txt_"${date}"_"${dirname}".txt; done This will concatenate the files in reverse order, which is what I actually need done (sorry if this wasn't clear).
