Skip to content

Instantly share code, notes, and snippets.

@lsauer
Created October 21, 2011 21:40
Show Gist options
  • Save lsauer/1305056 to your computer and use it in GitHub Desktop.
Save lsauer/1305056 to your computer and use it in GitHub Desktop.
Easiest way to find duplicate values in a JavaScript array - Native unique function implementation
//lo sauer, 2011; lsauer.com
/**
* I saw this thread: http://stackoverflow.com/questions/840781/easiest-way-to-find-duplicate-values-in-a-javascript-array
* The solutions above lacked the elegance that can be done a with map-reduce-like operations
* Since this implementation works with native functions, the speed is in most circumstances faster
* than any solution using scripted-logic
* Additionally, I needed to quickly filter duplicate url-entries for: http://lsauer.github.com/chrome-session-restore/
*/
//copy and paste: without error handling
Array.prototype.unique = function(){return this.sort().filter( function(v,i,o){if(i>=0 && v!==o[i-1]) return v;});}
//copy and paste: with error handling
Array.prototype.unique = function(){if(!(this instanceof Array))throw TypeError('Not an Array!'); return this.sort().filter( function(v,i,o){if(i>=0 && v!==o[i-1]) return v;});}
/**
* Numbers
*/
var arr = [324,3,32,5,52,2100,1,20,2,3,3,2,2,2,1,1,1];
//1. sorting / map
var a = arr.sort();
>>>[1, 1, 1, 1, 2, 2, 2, 2, 20, 2100, 3, 3, 3, 32, 324, 5, 52]
//2. reduce
//Note: if you need to copy the array at any point use Array.slice()
a.filter( function(v,i,o){if(i>=0 && v!==o[i-1]) return v;});
[2, 20, 2100, 3, 32, 324, 5, 52]
/**
* Strings
*/
var a = 'Magic belongs to Jerry Harry Jerry Harry Potter and Banana Joe'.split(' ');
a = a.sort()
>>>["Banana", "Harry", "Harry", "Jerry", "Jerry", "Joe", "Magic", "Potter", "and", "belongs", "to"]
a.filter( function(v,i,o){if(i>=0 && v!==o[i-1]) return v;});
["Harry", "Jerry", "Joe", "Magic", "Potter", "and", "belongs", "to"]
/**
* Additional information
*/
//you can also use the compact implementation used in is-lib ( github.com/lsauer/is-library )
a.filter( function(v,i,o){ return 1+i&&v!==o[i-1]?v:0;});
//...or a case insensitive function
a.filter( function(v,i,o){ return !i||v&&!RegExp(o[i-1],'i').test(v)});
//example
var a = 'Magic belongs to Jerry Harry Jerry JERRY AND HARRY Harry Potter and Banana Joe'.split(' ');
a.sort()
>>>["AND", "Banana", "HARRY", "Harry", "Harry", "JERRY", "Jerry", "Jerry", "Joe", "Magic", "Potter", "and", "belongs", "to"]
a.filter( function(v,i,o){ return i&&v&&!RegExp(o[i-1],'i').test(v)?v:0});
>>>["Banana", "HARRY", "JERRY", "Joe", "Magic", "Potter", "and", "belongs", "to"]
@bergus
Copy link

bergus commented May 14, 2012

@mflodin: Have you already seen my fork?

@mflodin
Copy link

mflodin commented May 14, 2012

@bergus Yeah, but I thought that was wrong too, remember? Turns out you had just forgotten to update the examples return lines (i.e. the >>> lines). But I actually ended up with basically the same functions as you anyway. I wish I had actually tried yours before discarding them based on the examples. =)

I just wanted to make @lsauer aware that his code might not work as expected.

@lsauer
Copy link
Author

lsauer commented May 14, 2012

Ok. To regard the index position 0, I could either remove 1+i&&.. or remove i&& altogether. 1+ to mflodin. thanks & cheers

I just saw my original filter function correctly stated 1+i&& all along, but got lost during its evolution in the altered filters underneath... go figure :).

@gsuttie
Copy link

gsuttie commented Feb 22, 2013

What about strings with numbers such as Zone1, Zone 1 & 2, Zone 1, 2 & 3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment