Skip to content

Instantly share code, notes, and snippets.

@gyli
Created July 9, 2018 19:51
Show Gist options
  • Save gyli/b6f19c4fde77c8f5d15caddbc72b3193 to your computer and use it in GitHub Desktop.
Save gyli/b6f19c4fde77c8f5d15caddbc72b3193 to your computer and use it in GitHub Desktop.
Find unique dict in a list in Python
import hashlib
def dict_hash(d: dict):
"""
Hash dictionary's key and value in string as the hash value of the dict
"""
out = hashlib.md5()
for key, value in d.items():
out.update(str(key).encode('utf-8'))
out.update(str(value).encode('utf-8'))
return out.hexdigest()
def sort_dict(item: dict):
"""
Sort nested dict. For Python >= 3.6
Example:
Input: {'a': 1, 'c': 3, 'b': {'b2': 2, 'b1': 1}}
Output: {'a': 1, 'b': {'b1': 1, 'b2': 2}, 'c': 3}
"""
return {k: sort_dict(v) if isinstance(v, dict) else v for k, v in sorted(item.items())}
def unique_dict_in_list(l: list, reorder: bool=False) -> list:
"""
Uniquify a list of dict
Example:
Input: [{'a': 1}, {'a': 1}, {'b': 2}]
Output: [{'a': 1}, {'b': 2}]
:param l: list of dictionaries
:param reorder: Whether to reorder each dict recursively.
It would be useful when there are same dict but declared in different orders, in Python >= 3.6.
"""
return list({dict_hash(d): d for d in (map(sort_dict, l) if reorder else l)}.values())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment