Skip to content

Instantly share code, notes, and snippets.

@mintchaos
Created July 27, 2012 18:25
Show Gist options
  • Save mintchaos/3189594 to your computer and use it in GitHub Desktop.
Save mintchaos/3189594 to your computer and use it in GitHub Desktop.
Uniquify list of dicts by a key.
"""
I have data like this.
I want to uniquify it based in the url. I have written code that works.
But it just seems gross. Is there a straightforward way to do this that I'm missing?
"""
embed_list = [
{
u'provider_url': u'http://twitter.com',
u'description': u'PIC: David and Samantha Cameron greet torch bearer Kate Nesbitt from the Royal Navy at 10 Downing St #olympictorch http://t.co/r5517bpB',
u'original_url': u'http://twitter.com/Number10gov/status/228538609618264065',
u'url': u'http://p.twimg.com/Ayvup5QCIAATPVl.jpg:large',
u'title': u'PIC: David and Samantha Cameron greet torch bearer Kate Nesbitt from the Royal Navy at 10 Downing St #olympictorch http://t.co/r5517bpB',
u'author_name': u'UK Prime Minister',
u'height': 400,
u'width': 540,
u'thumbnail_url': u'http://p.twimg.com/Ayvup5QCIAATPVl.jpg:thumb',
u'thumbnail_width': 150,
u'version': u'1.0',
u'provider_name': u'Twitter',
u'type': u'photo',
u'thumbnail_height': 150,
u'author_url': u'http://twitter.com/Number10gov'
},
{
u'provider_url': u'http://twitter.com',
u'description': u'PIC: David and Samantha Cameron greet torch bearer Kate Nesbitt from the Royal Navy at 10 Downing St #olympictorch http://t.co/r5517bpB',
u'original_url': u'http://t.co/r5517bpB',
u'url': u'http://p.twimg.com/Ayvup5QCIAATPVl.jpg:large',
u'title': u'PIC: David and Samantha Cameron greet torch bearer Kate Nesbitt from the Royal Navy at 10 Downing St #olympictorch http://t.co/r5517bpB',
u'author_name': u'UK Prime Minister',
u'height': 400,
u'width': 540,
u'thumbnail_url': u'http://p.twimg.com/Ayvup5QCIAATPVl.jpg:thumb',
u'thumbnail_width': 150,
u'version': u'1.0',
u'provider_name': u'Twitter',
u'type': u'photo',
u'thumbnail_height': 150,
u'author_url': u'http://twitter.com/Number10gov'
}
]
from itertools import groupby
# yes I know I need to sort this by the key first in order for groupby to consistently work.
embed_list = [list(g)[0] for k, g in groupby(embed_list, lambda x: x['url'])]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment