Skip to content

Instantly share code, notes, and snippets.

@Gustutu
Last active November 26, 2020 20:18
Show Gist options
  • Save Gustutu/cddf06f99e5a51c0043e93138de855fc to your computer and use it in GitHub Desktop.
Save Gustutu/cddf06f99e5a51c0043e93138de855fc to your computer and use it in GitHub Desktop.
Sync multiple ftp server to local filesystem in parallel
# requires sem, wget
# You must adapt paths especially for the wget command
declare -A ips
# dictionnary server name <--> IP
ips=(["AD0130"]="192.168.10.130" ["AD0136"]="192.168.10.136" ["AD0137"]="192.168.10.137" ["AD0138"]="192.168.10.138" ["AD0139"]="192.168.10.139" ["AD0140"]="192.168.10.140" ["AD0145"]="192.168.10.145" ["AD0146"]="192.168.10.146" ["AD0147"]="192.168.10.147" ["AD0148"]="192.168.10.148" ["AD0149"]="192.168.10.149")
for AD in "${!ips[@]}"
do
echo "$AD - ${ips[$AD]} "
echo $AD
# sync command generation
command="wget -a /var/log/syncFtp/syncFtp${AD}.log --timeout=10 --no-passive --tries=2 --ftp-password='' --ftp-user=user -m ftp://${ips[$AD]} -P /home/ubuntu/syncFolder/$AD"
echo $var
# Execution in a new process / thread?
# sem is an alias for parallel --semaphore. --semaphorename makes sure that only one task is executed per server
# more information by searching semaphore, sem man, parralel man on Google
sem --bg --semaphorename $AD $command
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment