Skip to content

Instantly share code, notes, and snippets.

@iyedb
Created March 16, 2014 13:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save iyedb/9583248 to your computer and use it in GitHub Desktop.
Save iyedb/9583248 to your computer and use it in GitHub Desktop.
sort files on remote host(s)
#!/bin/bash
REMOTE_HOST=freelaptop
if [ -d sortedfiles ]
then
rm -rf ./sortedfiles/*
else
tempdir=$(mktemp -d sortedfiles)
fi
if [ -f ./res ]
then
rm -rf res
fi
if [ -f ./sort_res ]
then
rm -rf ./sort_res
fi
for i in $*
do
(
scp "$i" iyed@$REMOTE_HOST:"$i" &> /dev/null
echo "$i,$REMOTE_HOST,$?" >> res
) &
done
wait
for l in $(cat res)
do
file=$(echo $l | cut -d, -f1)
host=$(echo $l | cut -d, -f2)
result=$(echo $l | cut -d, -f3)
if [ ! $result -eq 0 ]
then
echo "failed to copy $file on $host"
exit 1
fi
#alright all the files have been copied to a remote host
#we can start sorting each file of the remote host
done
for l in $(cat res)
do
file=$(echo $l | cut -d, -f1)
host=$(echo $l | cut -d, -f2)
result=$(echo $l | cut -d, -f3)
#ssh iyed@$host sort -n $file | cat > ./$tempdir/$file
(
ssh iyed@$host sort -n $file | cat > ./sortedfiles/$file
echo "$?,$file,$host" >> sort_res
)&
done
wait
for l in $(cat sort_res)
do
file=$(echo $l | cut -d, -f2)
host=$(echo $l | cut -d, -f3)
result=$(echo $l | cut -d, -f1)
if [ ! $result -eq 0 ]
then
echo "failed to sort $file on $host"
exit 1
fi
done
sort -n -m sortedfiles/* | grep -e '^$' -v
rm -rf ./res ./sort_res sortedfiles
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment