Skip to content

Instantly share code, notes, and snippets.

@ismet55555
Last active September 23, 2022 23:21
Show Gist options
  • Save ismet55555/60a64e472067b0fe63ab62f2db28afa5 to your computer and use it in GitHub Desktop.
Save ismet55555/60a64e472067b0fe63ab62f2db28afa5 to your computer and use it in GitHub Desktop.
Cleaning of secrets and sensitive information from git repository history

Git Repository Secret Cleaning

IMPORTANT: Be careful running these tools, git history will be changed

BFG Repo Cleaner

The BFG Repo Cleaner is a simpler, faster alternative to git-filter-branch for specifically cleansing bad data out of your Git repository history: Removing Crazy Big Files, Removing Passwords, Credentials & other Private data

  1. Close all pull requests on the git repo

  2. Clone the repo locally with the --mirror flag

    • git clone --mirror <REPO SSH TARGET>
  3. Download the BFG tool binary (bfg.jar)

  4. Create a replace.txt file that contains the sensitive text to be replaced by **REMOVED**

    • Each line in this text file is text to be replaced as shown below
    • # replace.txt
      mypassword
      someAPItoken$%
      login_info
  5. Run BFG Repo Cleaner

    • java -jar bfg.jar --replace-text replace.txt <REPO NAME>.git
  6. Cleanup unnecessary files and optimize the local repository

    • cd <REPO NAME>.git
    • git reflog expire --expire=now --all && git gc --prune=now --aggressive
  7. Push updated Git commits

    • git push --force
    • Ensure branch protection is off in repository settings

Git Repo Filter

git-filter-repo is a versitile tool to rewrite git history. It can do what BFG Repo Cleaner does and more.

  1. Close all pull requests on the git repo

  2. Clone the repo locally

    • git clone <REPO SSH TARGET>
  3. Change into repository directory

    • cd <REPO NAME>
  4. Install git-filter-repo

    • pip3 install git-filter-repo
  5. Create a replace.txt file that contains the sensitive text to be replaced by **REMOVED**

    • literal:MYSECRET==>**REMOVED**
    • More options for replace.txt: Example
  6. Run git-filter-repo with git

    • git filter-repo --replace-text replace.txt
  7. Push to remote

    • git push --force
    • Ensure branch protection is off in repository settings
    • May have to git remote add <REPO NAME> <REPO SSH TARGET>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment