Skip to content

Instantly share code, notes, and snippets.

@trex005
Last active August 29, 2015 14:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save trex005/571c1cdfbdb9fd3bf8e1 to your computer and use it in GitHub Desktop.
Save trex005/571c1cdfbdb9fd3bf8e1 to your computer and use it in GitHub Desktop.
/*
When the routine is running, the computer and video card resources will
be fully available and dedicated to the routine.
Images are located on a 1TB SSD + 1TB SSD Raid 0 Array
The Daemon must run in both Windows AND Linux.
*/
/* Start Image Deduplication Routine Daemon pseudo code */
//On open, will be passed the path to a project folder as argument -proj=path_to_folder
start(){
load (create if needed) database fingerprints from proj/.idr/fingerprints
//database fingerprints will include fingerprint_id, fingerprint, image_path.
//image_path will be relative to proj
//database fingerprints should be fully loaded into memory for reading, but all writes must
//be mirrored in memory and hard drive.
}
stop(){
Ensure all writes are complete to database fingerprints;
Unload database fingerprints;
shutdown program;
}
findDupe(image_path){
//Face Detection is suggested to be done with opencv
//We would like suggestions on libraries to use for image fingerprinting.
fingerprint = fingerprint(image_path);
if(fingerprint is error) return fingerprint;
compare found fingerprint to database fingerprints for duplicate
if(duplicate found){
return response of duplicate with original file and duplicate file paths;
} else {
flip image vertically;
fingerprint image again;
compare found fingerprint to database fingerprints for duplicate again;
if(duplicate found){
return response of duplicate with original file and duplicate file paths;
}
}
return response indicating no duplicate found;
}
fingerprint(image_path){
load image at image_path or return error ‘Unable to load image’;
find faces in loaded image;
if(number of faces in loaded image < 1){
return error ‘No Face Found’;
}
if(number of faces in loaded image > 1){
return error ‘Multiple Faces Found’;
}
isolate face in image;
rotate image to make eyes horizontal;
resize face to common size;
fingerprint isolated face in image;
return fingerprint;
}
removeImage(image_path){
delete image_path from database fingerprints
}
addImage(image_path){
fingerprint = fingerprint(image_path);
if(fingerprint is error) return fingerprint;
add image_path and fingerprint to database fingerprints;
}
/* End Image Deduplication Daemon pseudo code */
/* Start Image Deduplication GUI pseudo code */
On Open{
if(last_project_directory exists){
confirm with user they would like to load this project directory.
if(they would){
LoadProject(last_project_directory);
} else {
Prompt user for new project directory to load
}
}
}
LoadProject(project_directory){
start daemon with proj = project_directory;
save project_directory as last_project_directory in a way that it is available next time GUI starts.
create global variable proj = project_directory;
Create form with the options:
Start Deduping=>StartDeduping()
Clear Skip list=>ClearSkipList()
Exit => Exit();
}
StartDeduping(){
Load or create database skip_image from proj/.idr/skip_image_database;
//database skip_image will contain a list of all the file paths relative to
//proj that we have chosen to skip for now.
while(next_file = next file in ‘proj/Needs Deduplicated’ folder){
//Do magic here or in the daemon to fully use the available resources
if(next_file is not an image) continue;
if(next_file is in database skip_image) continue;
feed image to the daemon for deduplication callback to DupeResults(error,response); //findDupe(image_path)
}
notify user that deduplication has completed;
}
ClearSkipList(){
flush proj/.idr/skip_image_database;
flush database skip_image if in memory;
notify user that skip list has been cleared;
}
DupeResults(error,response){
if(error is ‘No Face Found’){
Move file to ‘proj/No Face Found’;
return;
} else if (error is ‘Multiple Faces Found’){
Move to ‘proj/Multiple Faces Found’;
return
} else if(error){
Log Error in proj/.idr/error_log;
return;
}
if(response indicates there is no duplicate){
Move file to proj/Deduplicated folder;
Instruct daemon to add file to fingerprints database;
return;
}
if(response includes duplicate file){
//file = the image file we were testing for deduplication
//duplicate_file = the duplicate that was found already in the database
display file and duplicate_file to user with options:
Delete{
if(deleting file){
move file to ‘proj/Duplicate’
close dialogue;
continue looping;
}
if(deleting duplicate_file){
move duplicate_file to ‘proj/Duplicate’
instruct daemon to remove duplicate_file and fingerprint from database fingerprints.
move file to ‘proj/Deduplicated’;
instruct daemon to add file and fingerprint to database fingerprints
}
}
Skip for now{
add image to database skip_image
close duplicate dialogue
}
Display image attributes (Dimensions, file type, file size, date, path);
Open in{
Open Dialogue to choose external program to open image in;
//should save the program used as it will almost always be the same program
}
}
}
Exit(){
Make sure database skip_image is saved to disk;
stop daemon;
}
/* End Image Deduplication GUI pseudo code */
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment