Back up GitLab to GitHub

gitlab-backup.md

Backup your GitLab repos to GitHub

GitLab recently decided to silently delete any repositories that hadn't been accessed in the last year. The announcement didn't go over well and they soon caved to public pressure and decided to instead back up inactive repos to object storage instead of unilaterally deleting them. I'm glad they reconsidered, but the experience left me with a bad taste in my mouth, so I decided to look into (relatively) low effort ways to transfer my repos out of GitLab.

The below is what I came up with. If people find it useful maybe I'll put it into a single script that can be run directly.

⚠️ NOTE: The below will back up your repos and any code within them, but does not back up other GitLab repository metadata like Issues or Merge Requests

Prereqs

To follow along you'll need the following installed:

• Homebrew (Mac or Linux) installed, or a copy of Steampipe, and the GitHub CLI installed via other means
• Both GitLab and GitHub configured for access over git+ssh

Backing up your repos

1. Create a GitLab personal access token

   go to https://gitlab.com/-/profile/personal_access_tokens and make an access token with at least read_api permissions

2. Copy the personal access token from GitLab and store it in an env var:

   export GITLAB_TOKEN="<your-token>"
   

3. Install Steampipe

   brew install turbot/tap/steampipe

4. Install the Steampipe GitLab plugin:

   steampipe plugin install theapsgroup/gitlab

5. Use Steampipe to retrieve all your project names and store them in the file projects.txt

   steampipe query --output csv --separator "\t" \
      'SELECT full_path FROM gitlab_my_project;' > projects.txt

   📝 NOTE: If you'd prefer to only backup projects that haven't had any activity in the last year you can replace the SELECT full_path... sql query above with this one:

   SELECT full_path FROM gitlab_my_project WHERE last_activity_at < date(now()) - interval '1 year';

6. Mirror each repo to a local folder (./gitlab-backup) :

   mkdir gitlab-backup; cd gitlab-backup
   cat projects.txt | while read p; do git clone --mirror "git@gitlab.com:${p}" ; done

7. Install the GitHub CLI (gh) if you don't have it, and make sure it's authenticated:

   brew install gh
   gh auth login

8. Create a private GitHub repo for each repo (you can remove the --private if you'd prefer to create public GitHub repos).

   The below will put a gitlab- prefix in front of each repos name in GitHub, if you'd prefer not to add the prefix or to use a different prefix change the gitlab_prefix="" variable below

   gitlab_prefix="gitlab-"
   ls | grep '\.git$' | while read r; do
      gh repo create --private "${gitlab_prefix}${r%.git}"
   done

9. Push each of your local repos up to GitHub

   gh_user="$(cat ~/.config/gh/hosts.yml | grep user: | sed 's/.*: //')"
   ls | grep '\.git$' | while read r; do
       echo ">> Mirroring repo '$r'"
       cd "$r"
       git remote set-url origin "git@github.com:${gh_user}/${gitlab_prefix}${r}"
       git push --mirror
       cd ..
   done

10. (optional) delete your local copy of all your GitLab repos now that they're backed up to GitHub

    cd ..
    rm -rf gitlab-backup