Skip to content

Instantly share code, notes, and snippets.

View robandpdx's full-sized avatar
🏠
Working from home

Rob Anderson robandpdx

🏠
Working from home
  • GitHub
  • Beaverton, Oregon
View GitHub Profile

Splitting large commits that prevent push to GitHub

You have discovered some very large commits in a repo history that prevent pushing the repo to GitHub, even when attempting to do a chunked push. These commits are larger that the push limit of 2G.

There are a few options for dealing with these large commits to unblock pushing to GitHub.

  1. delete the commit
  2. split the commit

Both options will rewrite the repo history. Deleting the commit can be problematic if there are later commits that depend on the commit being deleted. Spliting the commit into smaller chunks is a better option. We have created the chunk commit script to help commit the files from the large c

@robandpdx
robandpdx / svn-monorepo-to-git.md
Last active October 24, 2022 22:35
Moving an SVN monorepo to git

Moving an SVN monorepo to git

git svn expects your repo to be in the following structure...

./bigsvn
├── branches
│   ├── project1
│   └── project3
├── tags
│   └── project2
└── trunk
@robandpdx
robandpdx / monorepo-creation-with-git-subtree.md
Last active March 8, 2024 14:23
Making a monorepo from multiple git repos using git subtree

Making a monorepo from multiple git repos using git subtree

You may have several git repos that you want to combine into a single git repo. Here is how you can accomplish this using git subtree...

  1. Create a new git empty repo.
  2. Add a remote for each repo you want to include in the final repo git remote add k8s-azure-terraform https://github.com/robandpdx/k8s-azure-terraform.git.
  3. Fetch the remote branches git fetch k8s-azure-terraform
  4. Use git subtree add to pull in the repo git subtree add --prefix k8s-azure-terraform k8s-azure-terraform main
  5. Push all tags git push --tags

Migrating git repos without metadata

Metadata, like pull requests and issues, are not part of a git repo. These features are provided by the platform that hosts your shared git repo. There are tools to migrate metadata from BitBucket, GitLab, Azure DevOps, GitHub Enterprise Server, and github.com. If tools don't exist to migrate the metadata from the platform you are using, your only option is to migrate only the git repo. To migrate a git repo without medatadata, follow the instructions below.

Pre-migration

Familiarize yourself with the pre-migration documentation here.
If you are migrating a repo that uses LFS, make sure you have git-lfs installed.

Migration

  1. Clone the repo

LFS Migration

A note about OS

By default, windows and mac do not have a case sensitive filesystem. For this reason, I recommend using linux for lfs migration. Also, if the lfs migration seems to take a long time, this is often due to lots of disk I/O. To speed things up, use a cloud linux instance with max disk I/O.

Before migrating to LFS

The first step in migrating to LFS is finding what needs to be migrated. Use git-sizer for this task. Here is a utility script that can be used to run git-sizer on all repos in an org.

Another great tool for understanding blob sizes in a repo is git filter-repo. See these instructions for gathering blob sizing with git filter-repo.

@robandpdx
robandpdx / gh-gh-migration.md
Last active September 14, 2022 14:38
Migration instructions