Skip to content

Instantly share code, notes, and snippets.

@cosmoscalibur
Created June 6, 2017 15:32
Show Gist options
  • Save cosmoscalibur/8219a8d729d59b2e1c852c21b98927a2 to your computer and use it in GitHub Desktop.
Save cosmoscalibur/8219a8d729d59b2e1c852c21b98927a2 to your computer and use it in GitHub Desktop.
Remove duplicate files (use under your own risk). Example: `delrepeat "*.mp3"`
#! /usr/bin/env bash
## Usage: delrepeat STRING_PATTERN
## Example: delrepeat "*.mp3"
## IFS=$' \t\n' # default
SAVEIFS=$IFS
IFS=$'\n'
files=$(find . -iname "$1")
declare -a hashes
removed=0
for file in ${files[@]}; do
hash_file=$(md5sum $file | cut -d " " -f 1)
if [[ ${hashes[@]} == *$hash_file* ]]; then
echo "Removing $file."
rm $file
let removed=removed+1
else
hashes[${#hashes[@]}]=$hash_file
fi
done
echo "Removed $removed files."
IFS=$SAVEIFS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment