Skip to content

Instantly share code, notes, and snippets.

@lfoppiano

lfoppiano/find_duplicates.sh

Last active May 16, 2020 01:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lfoppiano/2bc14c96affc721f33234edb49cb5cfe to your computer and use it in GitHub Desktop.
Save lfoppiano/2bc14c96affc721f33234edb49cb5cfe to your computer and use it in GitHub Desktop.
Download ZIP
Find duplicated files
Raw
find_duplicates.sh
# Just return the sha of the duplicated files
sha1sum * | gsort | gawk '{a[$1]++}END{for(i in a){if(a[i]-1)print i, a[i]}}'
# Return the last file name for each duplicated files
sha1sum * | gsort | gawk '{a[$1]++; b[$1]=$2}END{for(i in a){if(a[i]-1)print i, b[i]}}'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment