Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
git pre-commit hook preventing large files

Usage

You can use in two ways.

  1. Directly as the pre-commit hook in your .git/hooks folder.

  2. With Husky by updating your package.json with:

"husky": {
    "hooks": {
      "pre-commit": "sh ./some-path/pre-commit-prevent-large-files.sh"
    }
}

Installation

@guysmoilov wrote an awesome installer script over here that means this hook will be added to all future repos you clone; how cool!

Credits

Based on @kiwidamien's original gist here

Alternatives

pre-commit is "a framework for managing and maintaining multi-language pre-commit hooks" and has a hook you can plug-in called: check-added-large-files. pre-commit is built with Python though, so you'll need Python installed.

#!/bin/bash
# This is a pre-commit hook that ensures attempts to commit files that are
# larger than $limit to your _local_ repo fail, with a helpful error message.
# Maximum file size limit in bytes
limit=$(( 5 * 2**20 )) # 5MB
limitInMB=$(( $limit / 2**20 ))
# Move to the repo root so git files paths make sense
repo_root=$( git rev-parse --show-toplevel )
cd $repo_root
empty_tree=$( git hash-object -t tree /dev/null )
if git rev-parse --verify HEAD > /dev/null 2>&1
then
against=HEAD
else
against="$empty_tree"
fi
# Set split so that for loop below can handle spaces in file names by splitting on line breaks
IFS='
'
echo "Checking staged file sizes"
shouldFail=false
# `--diff-filter=d` -> skip deletions
for file in $( git diff-index --cached --diff-filter=d --name-only "$against" ); do
# Skip for directories (git submodules)
if [[ -f "$file" ]]; then
file_size=$( ls -lan $file | awk '{ print $5 }' )
if [ "$file_size" -gt "$limit" ]; then
echo File $file is $(( $file_size / 2**20 )) MB, which is larger than our configured limit of $limitInMB MB
shouldFail=true
fi
fi
done
if $shouldFail
then
echo If you really need to commit this file, you can push with the --no-verify switch, but the file should definitely, definitely be under $limitInMB MB!!!
echo Commit aborted
exit 1;
fi
@guysmoilov

This comment has been minimized.

Copy link

@guysmoilov guysmoilov commented Dec 26, 2019

Hi, small fix, it should be:

against="$empty_tree"
@guysmoilov

This comment has been minimized.

Copy link

@guysmoilov guysmoilov commented Dec 26, 2019

Also, I wrote a small installer script for this, since I anticipate it could have wide usage: https://gist.github.com/guysmoilov/ddb3329e31b001c1e990e08394a08dc4
Thanks for you work!

@benmccallum

This comment has been minimized.

Copy link
Owner Author

@benmccallum benmccallum commented Jan 6, 2020

Thanks @guysmoilov, awesome work and thanks for sharing!

@Cyberbeni

This comment has been minimized.

Copy link

@Cyberbeni Cyberbeni commented Dec 8, 2020

file_size=$( ls -la $file | awk '{ print $5 }')
This doesn't work for groups with a space in them.
stat -f "%z" "$file" works on macOS but on Linux, it's stat -c "%s" "$file"
so the best is probably ls -lan $file | awk '{ print $5 }' which uses numerical value instead of user/group name

@Cyberbeni

This comment has been minimized.

Copy link

@Cyberbeni Cyberbeni commented Dec 8, 2020

The echo on line 39 doesn't use $limitInMB
Line 7 should be limit=$(( 5*2**20 ))
Line 8: limitInMB=$(( $limit / 2**20 ))
Line 32 should also use 2**20

@Cyberbeni

This comment has been minimized.

Copy link

@Cyberbeni Cyberbeni commented Dec 8, 2020

Also I think it would be better to use /bin/bash shebang: https://askubuntu.com/a/141932

@Cyberbeni

This comment has been minimized.

Copy link

@Cyberbeni Cyberbeni commented Dec 8, 2020

Line 3 ends with are and line 4 starts with are

@Cyberbeni

This comment has been minimized.

Copy link

@Cyberbeni Cyberbeni commented Jan 25, 2021

The current version will also print errors for deletions and will get messed up for submodules, this fixes that:

# `--diff-filter=d` -> skip deletions
for file in $( git diff-index --cached --diff-filter=d --name-only "$against" ); do
	# Skip for directories (git submodules)
	if [[ -f "$file" ]]; then
@benmccallum

This comment has been minimized.

Copy link
Owner Author

@benmccallum benmccallum commented Jan 26, 2021

Hey @Cyberbeni, thanks for the feedback. We don't actually use this at work anymore, I put a GitHub action in place instead, but your feedback seems valuable and in the interest of helping others I think we should apply your feedback here for sure.

@benmccallum

This comment has been minimized.

Copy link
Owner Author

@benmccallum benmccallum commented Jan 26, 2021

I've updated the gist with all your feedback everything except one I'm not sure about (see below). Do you mind checking it looks OK now to you?

Also I think it would be better to use /bin/bash shebang: https://askubuntu.com/a/141932

Do you mean in the example usage? e.g. it should be this?
"pre-commit": "!/bin/sh ./some-path/pre-commit-prevent-large-files.sh"

@benmccallum

This comment has been minimized.

Copy link
Owner Author

@benmccallum benmccallum commented Jan 26, 2021

cc: @guysmoilov, you might want to make these updates to your gist too

@Cyberbeni

This comment has been minimized.

Copy link

@Cyberbeni Cyberbeni commented Jan 26, 2021

I've updated the gist with all your feedback everything except one I'm not sure about (see below). Do you mind checking it looks OK now to you?

Also I think it would be better to use /bin/bash shebang: https://askubuntu.com/a/141932

Do you mean in the example usage? e.g. it should be this?
"pre-commit": "!/bin/sh ./some-path/pre-commit-prevent-large-files.sh"

No, I mean the first line in the file should be #!/bin/bash instead of #!/bin/sh because the latter can be different based on OS

@benmccallum

This comment has been minimized.

Copy link
Owner Author

@benmccallum benmccallum commented Feb 1, 2021

Thanks @Cyberbeni, updated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment