Skip to content

Instantly share code, notes, and snippets.

@amlamarra
Last active May 13, 2022 14:46
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save amlamarra/c6adda655101dc22f354a5569496752c to your computer and use it in GitHub Desktop.
Save amlamarra/c6adda655101dc22f354a5569496752c to your computer and use it in GitHub Desktop.
A Bash script to update multiple Git repositories at the same time. It also maintains a list of these repos in a text file to keep synced with your dotfiles across other systems.
#!/usr/bin/env bash
#########################################################################
## Title: reposync.sh
## Description: Syncs multiple git repositories in ~/tools/ based on the
## ~/.repolist file. Put each repo URL in there.
## Author: Andrew Lamarra
## Created: 01/23/2019
## Dependencies: bash (v4.0+), git
#########################################################################
# Setup
base=~/tools
list=.repolist
to_sync=~/$list
cd ~
urls=()
mkdir -p "$base"
# Get the list of Git repos in list.txt
readarray -t urls 2>/dev/null < $to_sync
# Get the list of directories in ~/tools/
dirs=( $(find $base -maxdepth 1 -mindepth 1 -type d) )
untracked=("${dirs[@]}")
function dir_not_found {
repo_url="$1"
echo "The following repository was found in your list but not in the $base/ directory:"
echo -e "\t$repo_url"
# Ask the user about cloning the repo (and repeat if input is invalid)
ans=''
while [[ $ans != 'y' ]] && [[ $ans != 'n' ]]; do
read -rp "Would you like to clone it? (Y/n) " ans
if [[ $ans == '' ]]; then ans="y"; fi
ans=$(echo "$ans" | tr '[:upper:]' '[:lower:]')
done
# Clone the repo if the user answers 'yes'
if [[ $ans == 'y' ]]; then
cd "$base"
git clone "$repo_url"
return
fi
# Ask the user if they'd like to remove that repo from the list
ans=''
while [[ $ans != 'y' ]] && [[ $ans != 'n' ]]; do
read -rp "Would you like to remove this repo from your $list file? (Y/n) " ans
if [[ $ans == '' ]]; then ans="y"; fi
ans=$(echo "$ans" | tr '[:upper:]' '[:lower:]')
done
# If 'yes' then delete the line from the list of repos
if [[ $ans == 'y' ]]; then
user=$(echo "$repo_url" | awk -F/ '{print $(NF-1)}' | cut -d: -f2)
repo=$(echo "$repo_url" | awk -F/ '{print $NF}')
sed -i "/$user\/$repo/d" $to_sync
fi
}
function repo_not_found {
repo_url="$1"
local_dir="$2"
echo "The following repository was found in $local_dir/ but not your list:"
echo -e "\t$repo_url"
# Ask the user about adding it to the list
ans=''
while [[ $ans != 'y' ]] && [[ $ans != 'n' ]]; do
read -rp "Would you like to add it to your $list file? (Y/n) " ans
if [[ $ans == '' ]]; then ans="y"; fi
ans=$(echo "$ans" | tr '[:upper:]' '[:lower:]')
done
if [[ $ans == 'y' ]]; then
echo "$repo_url" >> "$to_sync"
return
fi
# Ask the user if they'd like to remove that repo from the list
ans=''
while [[ $ans != 'y' ]] && [[ $ans != 'n' ]]; do
read -rp "Would you like to remove the repository ($local_dir)? (Y/n) " ans
if [[ $ans == '' ]]; then ans="y"; fi
ans=$(echo "$ans" | tr '[:upper:]' '[:lower:]')
done
# If 'yes' then delete the directory
if [[ $ans == 'y' ]]; then
rm -rf "$local_dir"
fi
}
# Iterate over the repos found in the list
for url in "${urls[@]}"; do
counter=0
for dir in "${dirs[@]}"; do
# Save the current number of elements in the dirs array
num_dirs="${#dirs[@]}"
cd "$dir"
if [[ $url == $(git config --get remote.origin.url) ]]; then
echo "Updating $(echo "$url" | awk -F/ '{print $NF}' | cut -d. -f1)..."
git pull
echo
# Remove the found directory
unset "untracked[$counter]"
break
fi
counter=$((counter + 1))
done
if [[ $counter -ge $num_dirs ]]; then
dir_not_found "$url"
echo
fi
done
# Iterate through leftover dirs to see if any are repos not in the list
for dir in "${untracked[@]}"; do
cd "$dir"
url=$(git config --get remote.origin.url)
if [[ $? == 0 ]]; then
repo_not_found "$url" "$dir"
echo
fi
done
@willinspire
Copy link

I'll be giving this script a test run here in a minute, but it looks really solid. Thanks for sharing.

@phatpaul
Copy link

Thanks for sharing, this gave me a good start for what I need. I would like to add a cmd-line option to answer yes-to-all for cloning from the server, i.e. ./reposync.sh --clone-all

@phatpaul
Copy link

Also a tip if you are cloning lots of private repos via HTTPS, tell git to store your password globally so you don't have to type it every time:
git config --global credential.helper store

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment