Skip to content

Instantly share code, notes, and snippets.

@vlazic
Forked from gianlucaparadise/big-repo-cloner.sh
Created August 12, 2020 12:17
Show Gist options
  • Save vlazic/b39e5d244fec86c4e001a3c4225e6bc1 to your computer and use it in GitHub Desktop.
Save vlazic/b39e5d244fec86c4e001a3c4225e6bc1 to your computer and use it in GitHub Desktop.
Big repo cloner - Script to clone a git repository bigger than 1GB
# Script hosted here: https://gist.github.com/gianlucaparadise/10286e0b1c5409bd1049d67640fb7c03
# A repository bigger than 1 GB can't be cloned normally.
# In order to clone it, you need to follow the instructions from:
# https://stackoverflow.com/a/53068021/6155481
# To execute:
# sh big-repo-cloner.sh <repo_uri> <repo_destination_folder> [<checkout_branch_name>]
# repo_uri and repo_destination_folder are mandatory
# checkout_branch_name is optional
# To run from web
# curl -sL https://git.io/JvtZ5 | sh -s repo_uri repo_folder
URL="$1" # mandatory
FOLDER="$2" # mandatory
BRANCH="$3" # optional
if [ -z "$URL" ]
then
echo "No Repository URL supplied"
exit 1
fi
if [ -z "$FOLDER" ]
then
echo "No Repository folder supplied"
exit 1
fi
# Shallow Clone
# First, turn off compression:
echo
echo "⏬ Cloning master without compression"
echo
git config --global core.compression 0
# Next, let's do a partial clone to truncate the amount of info coming down:
git clone --depth 1 "$URL" "$FOLDER"
echo
echo "⏬ Fetching rest of the clone"
echo
# When that works, go into the new directory and retrieve the rest of the clone:
cd "$FOLDER"
git fetch --unshallow
# or, alternately,
# git fetch --depth=2147483647
echo
echo "⏬ Pulling rest of the clone"
echo
# Now, do a pull:
git pull --all
echo
echo "⏬ Fixing git config"
echo
# Then to solve the problem of your local branch only tracking master
# open your git config file (.git/config) in the editor of your choice
# where it says:
# [remote "origin"]
#  url=<git repo url>
#  fetch = +refs/heads/master:refs/remotes/origin/master
# change the line
# fetch = +refs/heads/master:refs/remotes/origin/master
# to
# fetch = +refs/heads/*:refs/remotes/origin/*
sed -i '' -e 's/refs\/heads\/master/refs\/heads\/\*/g' .git/config
sed -i '' -e 's/refs\/remotes\/origin\/master/refs\/remotes\/origin\/\*/g' .git/config
echo
echo "⏬ Performing a full fetch"
echo
# Do a git fetch and git will pull all your remote branches now
git fetch
# I checkout the input branch only when supplied
if [ ! -z "$BRANCH" ]
then
echo
echo "⏬ Checking out $BRANCH"
echo
# Now I checkout our default branch
git checkout "$BRANCH"
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment