Skip to content

Instantly share code, notes, and snippets.

@sluongng
Last active October 28, 2021 13:31
Show Gist options
  • Save sluongng/3c2cac6560b3b8903a41a65625cb7613 to your computer and use it in GitHub Desktop.
Save sluongng/3c2cac6560b3b8903a41a65625cb7613 to your computer and use it in GitHub Desktop.
Monorepo sparse-checkout tricks

References

Tricks

Sparse checkout using CODEOWNERS

git clone --reference-if-able /on/disk/cache/repo/.git \
          --filter=tree:0 \
          --single-branch \
          --no-checkout \
          https://git.store.com/group/repo.git my-repo

cd my-repo

git sparse-checkout init --cone

git checkout master

grep $USER CODEOWNERS |\
  grep -v '^#' |\ # Ignore comments
  awk '{print $1}' |\ # Only print file path
  git sparse-checkout set --stdin

Sparse checkout using on-disk manifest

git clone --reference-if-able /on/disk/cache/repo/.git \
          --filter=tree:0 \
          --single-branch \
          --sparse \
          https://git.store.com/group/repo.git my-repo
          
cd my-repo

git sparse-checkout set dir/to/manifests

cat dir/to/manifests/*.manifest |\
  git sparse-checkout add --stdin

Generate sparse checkout manifest with bazel

  1. In a full-checkout copy, run bazel query
cd my-repo

bazel query --output=package 'buildfiles(deps(//projects/my-awsome-service/...))' |\
  awk '!/^@/ {print "/"$0}' \ # Ignore external deps, prepend all paths with '/'
  > dir/to/manifests/my-awsome-service/baze-deps.manifest
  
# Upload the file to some place
curl -XPOST dir/to/manifests/my-awsome-service/baze-deps.manifest

# Or add it into the git repo
git add dir/to/manifests/my-awsome-service/baze-deps.manifest
git commit -m 'manifest update' -m 'Update details: blah blah'
git push
  1. Checkout using bazel manifest
git clone --reference-if-able /on/disk/cache/repo/.git \
          --filter=tree:0 \
          --single-branch \
          --sparse \
          https://git.store.com/group/repo.git my-repo
          
cd my-repo

curl -s 'https://some-http-app/service/name/bazel-deps.manifest' |\
  git sparse-checkout set --stdin

Create and maintain on disk cache of repo

git clone --mirror https://git.store.com/group/repo.git /on/disk/cache/repo/.git

while true; do
  git -C /on/disk/cache/repo/.git fetch --prune --prune-tags origin;
  git -C /on/disk/cache/repo/.git gc --auto;
done

Using host local git cache in a container

# On baremetal/VM host
docker run --rm -it -v /on/disk/cache:/on/disk/cache:ro alpine /bin/bash

# Inside container
git clone --reference-if-able /on/disk/cache/repo/.git \
          --filter=tree:0 \
          --single-branch \
          --sparse \
          https://git.store.com/group/repo.git my-repo
@malomarrec
Copy link

malomarrec commented Oct 28, 2021

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment