Skip to content

Instantly share code, notes, and snippets.

@umbrant
Last active August 29, 2015 14:23
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save umbrant/cc66c859774e51447b6e to your computer and use it in GitHub Desktop.
Save umbrant/cc66c859774e51447b6e to your computer and use it in GitHub Desktop.
Program to benchmark hardlink creation using rsync
#!/usr/bin/env bash
function p {
echo `date --rfc-3339=ns` "$@"
}
BASEDIR=`pwd`/hardtest
SOURCEDIR=$BASEDIR/dfs
SNAPDIR=$BASEDIR/dfs-snap
LINKDIR=$BASEDIR/dfs-link
NUMDIRS=12
NUMSUBDIRS=100
NUMFILES=100
p "Creating subdir and subfile hierarchy"
for i in `seq 1 $NUMDIRS`; do
for j in `seq 1 $NUMSUBDIRS`; do
mkdir -p $SOURCEDIR/subdir$i/subdir$j
for k in `seq 1 $NUMFILES`; do
touch $SOURCEDIR/subdir$i/subdir$j/file$k &
wait
done
wait
done
p "Created subdir $i of $NUMDIRS"
done
p "Creating initial snapshot for rsync"
rsync -a $SOURCEDIR/ $SNAPDIR/
p "Creating hardlinks via rsync --link-dest"
START=$(date '+%s%N')
for i in `seq 1 $NUMDIRS`; do
mkdir -p $LINKDIR/subdir$i
rsync -a --link-dest=$SNAPDIR/subdir$i/ $SOURCEDIR/subdir$i/ $LINKDIR/subdir$i/ &
done
wait
END=$(date '+%s%N')
p "Finished making hardlinks"
NUMLINKS=`find $LINKDIR -type f | wc -l`
DURATION=$(expr $END - $START)
RATE=`echo "$NUMLINKS/($DURATION/1000000000)" | bc -l`
p "Number of links: " $NUMLINKS
p "Hardlink time (ns): " $DURATION
p "Throughput (links/s): " $RATE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment