robandpdx

Documentation

GitHub CLI
GitHub Enterprise Importer CLI
GitHub Enterprise Importer (GEI)
GEI limitations

The following is a check-list for steps to take, before and after migrations.
Before Migration
Organization / Users:

• EMU organizations created (named differently than source organizations)

robandpdx

LFS Migration

A note about OS

By default, windows and mac do not have a case sensitive filesystem. For this reason, I recommend using linux for lfs migration. Also, if the lfs migration seems to take a long time, this is often due to lots of disk I/O. To speed things up, use a cloud linux instance with max disk I/O.

Before migrating to LFS

The first step in migrating to LFS is finding what needs to be migrated. Use git-sizer for this task. Here is a utility script that can be used to run git-sizer on all repos in an org.

Another great tool for understanding blob sizes in a repo is git filter-repo. See these instructions for gathering blob sizing with git filter-repo.

robandpdx

Migrating git repos without metadata

Metadata, like pull requests and issues, are not part of a git repo. These features are provided by the platform that hosts your shared git repo. There are tools to migrate metadata from BitBucket, GitLab, Azure DevOps, GitHub Enterprise Server, and github.com. If tools don't exist to migrate the metadata from the platform you are using, your only option is to migrate only the git repo. To migrate a git repo without medatadata, follow the instructions below.

Pre-migration

Familiarize yourself with the pre-migration documentation here.
If you are migrating a repo that uses LFS, make sure you have git-lfs installed.

Migration

1. Clone the repo

robandpdx

Making a monorepo from multiple git repos using git subtree

You may have several git repos that you want to combine into a single git repo. Here is how you can accomplish this using git subtree...

1. Create a new git empty repo.
2. Add a remote for each repo you want to include in the final repo git remote add k8s-azure-terraform https://github.com/robandpdx/k8s-azure-terraform.git.
3. Fetch the remote branches git fetch k8s-azure-terraform
4. Use git subtree add to pull in the repo git subtree add --prefix k8s-azure-terraform k8s-azure-terraform main
5. Push all tags git push --tags

robandpdx

Moving an SVN monorepo to git

git svn expects your repo to be in the following structure...

./bigsvn
├── branches
│   ├── project1
│   └── project3
├── tags
│   └── project2
└── trunk

robandpdx

Splitting large commits that prevent push to GitHub

You have discovered some very large commits in a repo history that prevent pushing the repo to GitHub, even when attempting to do a chunked push. These commits are larger that the push limit of 2G.

There are a few options for dealing with these large commits to unblock pushing to GitHub.

1. delete the commit
2. split the commit

Both options will rewrite the repo history. Deleting the commit can be problematic if there are later commits that depend on the commit being deleted. Spliting the commit into smaller chunks is a better option. We have created the chunk commit script to help commit the files from the large c

robandpdx

PyTorch workflow testing

In order to iterate faster on workflows it is necessary to setup an environment to run the test.

1. Launch a g5.4xlarge instance in aws with Deep Learning AMI GPU PyTorch 1.10.0 (Ubuntu 20.04) 20221003 ami-09b3fda578ad20138
2. Generate an ssh key pair and add it to your github account.
3. Clone pytorch/rl repo to /home/ec2-user/github/rl
4. Use the following script to start the container and get a shell...

#!/bin/bash

robandpdx

Stacked diff workflow

ghstack

ghstack
Stacked diffs and ghstack
Ghstack is a little rough around the edges. It seems to work as advertised, but some of the ways it works are a bit quirky. The following is a list of things I don't like about ghstack:

• it creates 3 branches for each commit
• PR are from head->base rather than from branch->master
• you need force push access to main in order to land a PR
• branch protection rules are useless, need to be bypassed by force pushes

robandpdx

Migration Guide from Gerrit to GitHub

Scope of the document

This document provides the steps involved in migrating source code from Gerrit to GitHub. The CI/CD setup part is currently out of scope of this document.

Introduction

Below is a guided procedure to ensure a migration from Gerrit to GitHub.

Migration steps

The following steps act as high-level phases when implementing a migration project:

robandpdx

These instructions explain how to migrate a repo from GitHub Enterprise Server to GitHub Enterprise Cloud using the gh gei CLI when either:

• (a) you want to use an unsupported blob storage provider; or
• (b) you want to use a supported blob storage provider with different authentication

The following instructions assume that you have curl and jq installed. There are available for Linux, macOS and Windows Subsystem for Linux (WSL).

1. Make a note of your GitHub Enterprise Server hostname (e.g. github.acmecorp.com) and expose it as the GHES_HOST environment variable: export GHES_HOST=github.acmecorp.com.
2. Make a note of the GitHub Enterprise Server organization that owns your origin repo, and expose it as the GHES_ORGANIZATION environment variable: export GHES_ORGANIZATION=engineering.
3. Make a note of the name of the repo you are migrating in GitHub Enterprise Server, and expose it as the GHES_REPO environment variable: export GHES_REPO=webapp.