Skip to content

Instantly share code, notes, and snippets.

@kcuzner kcuzner/archive.sh
Created Oct 31, 2012

Embed
What would you like to do?
Multi-process folder archiving
#!/bin/bash
# purpose: to compress my home folder
#output folder for the files
OUTPUTDIR="/media/My Book/HomeFolderBackup/"
#number of compressing process
NPROC=6
#queue stuff
QUEUE=""
NUM=0
function queue {
QUEUE="$QUEUE $1"
NUM=$(($NUM+1))
}
function regeneratequeue {
OLDREQUEUE=$QUEUE
QUEUE=""
NUM=0
for PID in $OLDREQUEUE
do
if [ -d /proc/$PID ]; then
QUEUE="$QUEUE $PID"
NUM=$((NUM + 1))
fi
done
}
function checkqueue {
OLDCHQUEUE=$QUEUE
for PID in $OLDCHQUEUE
do
if [ ! -d /proc/$PID ]; then
regeneratequeue
break
fi
done
}
#put the list of files into a file
ls -l > archiveList.txt
#and loop through
I=0
cat archiveList.txt | while read line
do
FNAME=$(echo ${line} | awk '{ print $8 }')
tar pcvzf "${OUTPUTDIR}${FNAME}.tar.gz" "${FNAME}" &
PID=$!
queue $PID
while [ $NUM -ge $NPROC ]; do
checkqueue
sleep 0.4
done
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.