thomassajot

Git clean history from large files and secrets

To remove files, we are going to use BFG, which is simpler and less prone to error compared to git filter-branch.

Finding large files:

The following command finds the blob size of all objects (blob or tree) in the index. We sort the blob by their size (the 3rd column) and save them to a file.

git verify-pack -v .git/objects/pack/pack-<SOME_ID>.idx | grep -v chain | sort -k3n > ../git_verify_pack_all.tsv

thomassajot

import json
import subprocess

import pandas as pd
from tensorflow import __version__

def run_cmd(cmd):
    cmd = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    return cmd