sarah-j-smith/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Cleaning Secrets from a Repo

FIRST: Revoke the secrets from whatever assets or platform they provide access to

Twitter
Google

Assume that the keys have already fallen in to the hands of bad actors, and
immediately change the locks so that the keys are no use.
Cleaning House

HAVE YOU REALLY REVOKED ALL THE KEYS?
You may want to clean up the repo where the keys were exposed to remove traces
of the keys (which you have now rendered useless) because:

Other authors in the repo might find the keys and mistakenly issue & expose new ones
To protect the guilty - cover your tracks - as an act of contrition

Note that cleaning house like this will not remove the keys from any fork or offline copy
someone may (read as "already has") taken of the repo.
Install BFG


BFG Repo Cleaner

curl -O https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar
mkdir -p ~/bin
mv -f bfg-1.14.0.jar ~/bin/.
java -jar ~/bin/bfg-1.14.0.jar

If the last command didn't work, check you have a Java JDK in your path.  Hint: if you have an Android SDK installed
on Mac then this might work:
.bash_profile:
# Java from Android SDK
export JAVA_HOME="/Applications/Android Studio.app/Contents/jre/jdk/Contents/Home"
export PATH="${JAVA_HOME}/bin:${PATH}"
Otherwise install a Java JDK and try again.
Get Ready for the BFG


Close all PR's and get rid of any stale branches
Stop any CI/CD activity or anything that could update your repo

Clean out the secrets

Make a fresh clone of your repo

mkdir -p ~/src && cd ~/src
git clone git@github.com:my-git-name/RepoWithExposedSecrets.git freshRepoWithExposedSecrets

Remove the file containing the secrets

cd freshRepoWithExposedSecrets
git rm Credentials.json
git commit -m "Remove credentials file"

Add another commit so the secret removal is not on HEAD


This is important - the BFG cannot clean HEAD commit history
If you just removed a sensitive file then the commit will show the secret

echo "Please never add secrets to this repo" >> README.md
git add README.md
git commit -m "Document secrets policy"


Push the results

git push

Run the BFG

cd ~/src
git clone --mirror git@github.com:my-git-name/RepoWithExposedSecrets.git
java -jar ~/bin/bfg-1.14.0.jar --delete-files Credentials.json RepoWithExposedSecrets.git

After the above you should get output like:
~/src $ java -jar ~/bin/bfg-1.14.0.jar --delete-files Credentials.json RepoWithExposedSecrets.git

Using repo : /Users/my-git-name/src/RepoWithExposedSecrets.git

Found 23 objects to protect
Found 4 commit-pointing refs : HEAD, refs/heads/ex0, refs/heads/ex1, refs/heads/master

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

 * commit 593dbab8 (protected by 'HEAD')

Cleaning
--------

Found 29 commits
Cleaning commits:       100% (29/29)
Cleaning commits completed in 74 ms.

Updating 1 Ref
--------------

	Ref                 Before     After   
	---------------------------------------
	refs/heads/master | 594dbcb8 | 033618fc

Updating references:    100% (1/1)
...Ref update completed in 11 ms.

Commit Tree-Dirt History
------------------------

	Earliest               Latest
	|                           |
	.....................DmDmDDmm

	D = dirty commits (file tree fixed)
	m = modified commits (commit message or parents changed)
	. = clean commits (no changes to file tree)

	                        Before     After   
	-------------------------------------------
	First modified commit | 56be179e | 127e3e8b
	Last dirty commit     | edad0d34 | 1cfe71d0

Deleted files
-------------

	Filename                  Git id           
	-------------------------------------------
	Credentials.json        | 453b2544 (117 B )


In total, 9 object ids were changed. Full details are logged here:

	/Users/my-git-name/src/RepoWithExposedSecrets.git.bfg-report/2021-07-18/15-22-55

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive

If the output says something about protected commits and zero changes then you have a protected
commit that is blocking the BFG from cleaning your repo.
Most likely you did not add an additional commit.  Check above where it says:

Add another commit so the secret removal is not on HEAD.

More details can be read on SO, but seriously its as simple as making sure that

the commit that removes the file containing the secrets
is not the most recent.

If you get the above you will need to remove the mirror repo & fresh clone and start from the top of this document.
cd ~/src
rm -Rf RepoWithExposedSecrets.git
rm -Rf freshRepoWithExposedSecrets


Go to the top and start again

Push the BFG results

OK - it worked?  The BFG should have printed out a line like:
BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive

...copy the run part and execute it in the mirror repo:
cd ~/src/RepoWithExposedSecrets.git
git reflog expire --expire=now --all && git gc --prune=now --aggressive

You should see something like this:
Enumerating objects: 139, done.
Counting objects: 100% (139/139), done.
Delta compression using up to 16 threads
Compressing objects: 100% (138/138), done.
Writing objects: 100% (139/139), done.
Building bitmaps: 100% (29/29), done.
Total 139 (delta 89), reused 47 (delta 0), pack-reused 0

Now push the cleaned repo up to your mainline:
git push

You may need to do a force push, or git may figure that out. What you should see on push is:
Enumerating objects: 38, done.
Counting objects: 100% (38/38), done.
Writing objects: 100% (108/108), 16.04 KiB | 16.04 MiB/s, done.
Total 108 (delta 38), reused 38 (delta 38), pack-reused 70
remote: Resolving deltas: 100% (77/77), completed with 13 local objects.
To github.com:sarah-j-smith/QTweetGettr.git
 + 594dbab...033628f master -> master (forced update)

Go to the Repo on Github and check


Check the relevant file is gone
Check that commit history with the diff showing the secrets is not visible