Skip to content

Instantly share code, notes, and snippets.

@JonasGroeger
Last active November 4, 2023 13:10
Show Gist options
  • Star 63 You must be signed in to star a gist
  • Fork 31 You must be signed in to fork a gist
  • Save JonasGroeger/1b5155e461036b557d0fb4b3307e1e75 to your computer and use it in GitHub Desktop.
Save JonasGroeger/1b5155e461036b557d0fb4b3307e1e75 to your computer and use it in GitHub Desktop.
Gitlab: Clone / Pull all projects in a group
#!/usr/bin/env bash
# Documentation
# https://docs.gitlab.com/ce/api/projects.html#list-projects
NAMESPACE="YOUR_NAMESPACE"
BASE_PATH="https://gitlab.example.com/"
PROJECT_SEARCH_PARAM=""
PROJECT_SELECTION="select(.namespace.name == \"$NAMESPACE\")"
PROJECT_PROJECTION="{ "path": .path, "git": .ssh_url_to_repo }"
if [ -z "$GITLAB_PRIVATE_TOKEN" ]; then
echo "Please set the environment variable GITLAB_PRIVATE_TOKEN"
echo "See ${BASE_PATH}profile/account"
exit 1
fi
FILENAME="repos.json"
trap "{ rm -f $FILENAME; }" EXIT
curl -s "${BASE_PATH}api/v3/projects?private_token=$GITLAB_PRIVATE_TOKEN&search=$PROJECT_SEARCH_PARAM&per_page=999" \
| jq --raw-output --compact-output ".[] | $PROJECT_SELECTION | $PROJECT_PROJECTION" > "$FILENAME"
while read repo; do
THEPATH=$(echo "$repo" | jq -r ".path")
GIT=$(echo "$repo" | jq -r ".git")
if [ ! -d "$THEPATH" ]; then
echo "Cloning $THEPATH ( $GIT )"
git clone "$GIT" --quiet &
else
echo "Pulling $THEPATH"
(cd "$THEPATH" && git pull --quiet) &
fi
done < "$FILENAME"
wait
@MeenalMis
Copy link

Hey @JonasGroeger , when I tried using your script, got an error of "parse error: Invalid numeric literal at line 1, column 16". Not sure exactly what Iam giving wrong, here is the snippet of my code -

#!/usr/bin/env bash

Documentation

https://docs.gitlab.com/ce/api/projects.html#list-projects

NAMESPACE="fsgbu-pbsm/fsgbu-ofspacs/dimension-management"
GITLAB_PRIVATE_TOKEN="xxxx7fcxx32xxxxw4xx"
BASE_PATH="https://cloudlab.us.oracle.com/"
PROJECT_SEARCH_PARAM=""
PROJECT_SELECTION="select(.namespace.name == "$NAMESPACE")"
PROJECT_PROJECTION="{ "path": .path, "git": .ssh_url_to_repo }"

if [ -z "$GITLAB_PRIVATE_TOKEN" ]; then
echo "Please set the environment variable GITLAB_PRIVATE_TOKEN"
echo "See ${BASE_PATH}profile/account"
exit 1
fi

FILENAME="repos.json"
............

Note-> I need to pull all subgroups under group dimension-management.

@adroste
Copy link

adroste commented Nov 17, 2022

I wanted to clone also subgroups while keeping the tree structure. Since none of the solutions worked flawless, I wrote my own simple script => https://github.com/adroste/gitlab-clone-group

@Anjing1993
Copy link

Anjing1993 commented Nov 17, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment