Skip to content

Instantly share code, notes, and snippets.

@koumaza
Created November 16, 2020 01:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save koumaza/8f8f59c1b377573e42ac82613975f8f7 to your computer and use it in GitHub Desktop.
Save koumaza/8f8f59c1b377573e42ac82613975f8f7 to your computer and use it in GitHub Desktop.
For keybase filesystem (kbfs) mirroring
#!/usr/bin/env bash
# EX
# bash parasync.bash /run/user/1001/keybase/kbfs/team/team/ ~/mnt/ 10 /run/user/1001/keybase/kbfs/team/team/ -rvWtS --delete --force --progress --partial
Usage() {
#Help
cat <<'EOF'
./parasync.bash [from] [to] [parallels] [source include_dir] [rsync option]
Number of parallels:
def | default (10)
num | set to num
rsync option:
'--relative --stats --safe-links --ignore-existing --human-readable'
are already specfied.
Requrements:
- bash
- rsync
- parallel
EOF
}
# Env
export TEMP_DIR="$(mktemp -d)"
echo "TEMPDIR: $TEMP_DIR"
if [ "$3" = "def" ];then
export PARA_NUM=10
else
export PARA_NUM=$3
fi
for n in $(seq 5 $#)
do
export RSYNC_OPT="${RSYNC_OPT} $(eval echo -n '$'$n)"
done
# Help
if [ "$1" = "-h" ];then
Usage ; exit 0
elif [ "$1" = "--help" ];then
Usage ; exit 0
elif [ -z "$1" ];then
echo 'Argument needed.' ; exit 1
fi
Prompt() {
#Run
if [ "$PARASYNC_SKIP_PROMPT" = "true" ];then
:
else
read -p "Are you sure? | Y/n > " sttprompt
if [ "$sttprompt" = "n" ];then
echo 'Aborted' ; exit 0
fi
fi
}
# Run
export FROMDIR="$PWD"
cd $4
rsync $(echo -n $RSYNC_OPT|tr ' ' ' ') --stats --safe-links --ignore-existing --dry-run --human-readable \
$1 $2 | tee $TEMP_DIR/queue.list
Prompt
echo $PWD
cat $TEMP_DIR/queue.list | \
parallel --will-cite -j 6 rsync $(echo -n $RSYNC_OPT|tr ' ' ' ') --relative --stats --safe-links --ignore-existing --human-readable {} \
$2 | tee $TEMP_DIR/result.log
cd $FROMDIR
@koumaza
Copy link
Author

koumaza commented Nov 16, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment