Reduce duplicated images into md5 named image bank with symlinks from original location

README.md

Image Banker

Method to de-duplicate a large amount of images contained in a complicated folder structure which must be maintained.

Given a directory containing duplicate jpg images with any folder structure, img-banker.sh will:

1. Recursively find all jpgs.
2. Calculate the file's md5 hash.
3. Copy the file to img-bank directory using the md5 hash as the new name (the "banked image").
4. Replace the original image with a symlink to the banked image.

img-banker-test.sh

#!/bin/bash

# build html page for testing

rm img-bank-test.html

multiple_cmd() { 
  echo "<img src='$1' />"  >> img-bank-test.html
}; 
export -f multiple_cmd; 
find ./repo -name '*.jpg' -exec bash -c 'multiple_cmd "$0"' {} \;

img-banker.sh

#!/bin/bash

bank_images() { 
  IMGBANKDIR=~/IMAGE-BANK-TEST/img-bank/
  ORIGNAME=$1
  MD5NAME=$(md5 "$1" | awk '{ print $4 }').jpg
  cp -v $1 $IMGBANKDIR$MD5NAME
  if [ $? -eq 0 ]; then
    rm $1
    ln -s $IMGBANKDIR$MD5NAME $ORIGNAME
  else
      echo FAIL
  fi

}; 
export -f bank_images; 
find ~/IMAGE-BANK-TEST/repo -type f -name '*.jpg' -exec bash -c 'bank_images "$0"' {} \;