Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
# This is a really old post, in the comments (and stackoverflow too) you'll find better solutions.
def find(key, dictionary):
for k, v in dictionary.iteritems():
if k == key:
yield v
elif isinstance(v, dict):
for result in find(key, v):
yield result
elif isinstance(v, list):
for d in v:
for result in find(key, d):
yield result
example = {'app_url': '', 'models': [{'perms': {'add': True, 'change': True, 'delete': True}, 'add_url': '/admin/cms/news/add/', 'admin_url': '/admin/cms/news/', 'name': ''}], 'has_module_perms': True, 'name': u'CMS'}
list(find('admin_url', example))
@semolex

This comment has been minimized.

Copy link

@semolex semolex commented Apr 26, 2015

Great thing! Thanks!

@chinmayb

This comment has been minimized.

Copy link

@chinmayb chinmayb commented Jul 3, 2015

nicely done

@russellballestrini

This comment has been minimized.

Copy link

@russellballestrini russellballestrini commented Nov 10, 2015

I took this code, modified it a bit to:

@daxonwax

This comment has been minimized.

Copy link

@daxonwax daxonwax commented May 16, 2016

beautiful. thankyou.

@robfrei

This comment has been minimized.

Copy link

@robfrei robfrei commented May 28, 2016

Very nice. Thanks!

@abdullahaftab

This comment has been minimized.

Copy link

@abdullahaftab abdullahaftab commented Jan 29, 2017

Perfect !

@Aspen129

This comment has been minimized.

Copy link

@Aspen129 Aspen129 commented May 23, 2017

This is exactly what I needed to deal with nested dict's from an API request; thank you!!

@smopucilowski

This comment has been minimized.

Copy link

@smopucilowski smopucilowski commented Jun 16, 2017

for item in iterator: yield item can be replaced with yield from iterator.

@takenoto

This comment has been minimized.

Copy link

@takenoto takenoto commented Jan 22, 2018

Many thanks 👍 !!!

@ajay-manwani

This comment has been minimized.

Copy link

@ajay-manwani ajay-manwani commented Mar 2, 2018

Thanks a Lot !!

@truongphan

This comment has been minimized.

Copy link

@truongphan truongphan commented Mar 20, 2018

thanks you so much

@alon-unifi

This comment has been minimized.

Copy link

@alon-unifi alon-unifi commented Apr 2, 2018

Nice! if any of your nested lists contains primitives (e.g, strings) and not always dicts, it will error out. In that case add a check for it and skip the element (added line 2 before last):

def find(key, dictionary):
    for k, v in dictionary.iteritems():
        if k == key:
            yield v
        elif isinstance(v, dict):
            for result in find(key, v):
                yield result
        elif isinstance(v, list):
            for d in v:
                if isinstance(d, dict):
                    for result in find(key, d):
                        yield result
@jenweber

This comment has been minimized.

Copy link

@jenweber jenweber commented Apr 17, 2018

Python3 users should replace dictionary.iteritems() with dictionary.items(). iteritems was removed.

Thanks for sharing!

@dannyblaker

This comment has been minimized.

Copy link

@dannyblaker dannyblaker commented May 4, 2018

Awesome thanks

@Maxhirez

This comment has been minimized.

Copy link

@Maxhirez Maxhirez commented Jun 18, 2018

Nice work-just what I needed!

@treblekicker

This comment has been minimized.

Copy link

@treblekicker treblekicker commented Jun 20, 2018

Beautiful, thank you!

@abhiram24

This comment has been minimized.

Copy link

@abhiram24 abhiram24 commented Aug 9, 2018

Super sweet and clean. Thank you!

For Python 3.x users, just replace " for k, v in dictionary.iteritems():" with " for k, v in dictionary.items():"

@PatrikHlobil

This comment has been minimized.

Copy link

@PatrikHlobil PatrikHlobil commented Sep 24, 2018

Thanks for your nice example. I changed it a bit for my use case, where I needed to get all contained keys of a nested iterable as a list (also for the values) as can be seen in https://gist.github.com/PatrikHlobil/9d045e43fe44df2d5fd8b570f9fd78cc :

def iterate_all(iterable, returned="key"):
    
    """Returns an iterator that returns all keys or values
       of a (nested) iterable.
       
       Arguments:
           - iterable: <list> or <dictionary>
           - returned: <string> "key" or "value"
           
       Returns:
           - <iterator>
    """
  
    if isinstance(iterable, dict):
        for key, value in iterable.items():
            if returned == "key":
                yield key
            elif returned == "value":
                if not (isinstance(value, dict) or isinstance(value, list)):
                    yield value
            else:
                raise ValueError("'returned' keyword only accepts 'key' or 'value'.")
            for ret in iterate_all_keys(value, returned=returned):
                yield ret
    elif isinstance(iterable, list):
        for el in iterable:
            for ret in iterate_all_keys(el, returned=returned):
                yield ret

For the example:

example = {'app_url': '', 'models': [{'perms': {'add': True, 'change': True, 'delete': True}, 
           'add_url': '/admin/cms/news/add/', 'admin_url': '/admin/cms/news/', 'name': ''}], 
           'has_module_perms': True, 'name': u'CMS'}

It gives the following output:

list(iterate_all(example, "key"))
>>> ['app_url', 'models', 'perms', 'add', 'change', 'delete', 'add_url', 'admin_url', 'name', 'has_module_perms', 'name']

list(iterate_all(example, "value"))
>>>['', True, True, True, '/admin/cms/news/add/', '/admin/cms/news/', '', True, 'CMS']
@agustinscigliano

This comment has been minimized.

Copy link

@agustinscigliano agustinscigliano commented Sep 25, 2018

Is there a way to respect the hierarchy? So for example, if i have:

[
    {
        "id": 800,
        "children": [
            {
                "id": 801,
                "children": [
                    {
                        "id": 804,
                    },
                    {
                        "id": 805,
                     }
    },
]

I get: 800/801/804 , 800/801/805 (and every combination possible)

@rameshrvr

This comment has been minimized.

Copy link

@rameshrvr rameshrvr commented Dec 3, 2018

@PatrikHlobil You can use the module nested_lookup for fetching the keys(https://pypi.python.org/pypi/nested-lookup).
Example:

>>> example = {'app_url': '', 'models': [{'perms': {'add': True, 'change': True, 'delete': True}, 
...            'add_url': '/admin/cms/news/add/', 'admin_url': '/admin/cms/news/', 'name': ''}], 
...            'has_module_perms': True, 'name': u'CMS'}
>>> from nested_lookup import get_all_keys
>>> get_all_keys(example)
['app_url', 'models', 'perms', 'add', 'change', 'delete', 'add_url', 'admin_url', 'name', 'has_module_perms', 'name']
>>> 
@praveenbandi86

This comment has been minimized.

Copy link

@praveenbandi86 praveenbandi86 commented Dec 27, 2018

Can you pl give me the explanation of the code.

@vaibhavQ

This comment has been minimized.

Copy link

@vaibhavQ vaibhavQ commented Feb 25, 2019

Hey, can you give the code for deleting all the entries in dict matching the key we send.

@Sunil-ghodela

This comment has been minimized.

Copy link

@Sunil-ghodela Sunil-ghodela commented Mar 3, 2019

Thanks...

@AvdN

This comment has been minimized.

Copy link

@AvdN AvdN commented Apr 10, 2019

This is broken, except for the limited example you give where you have a dict at the root of your data-structure and restrictions on lists.
If you just add another key to your example:

example['xyz'] = [[dict(admin_url="broken")]]

You don't get two results, as one would expect, you get an attributeerror.

That StackOverflow post has much better answers, you should not indiscriminately copy the one with the unrealistic assumptions. You should also indicate this is Python2 only because of your use of .iteritems()

@Palash90

This comment has been minimized.

Copy link

@Palash90 Palash90 commented Dec 19, 2019

Thank you very much, this saved my day.

@futoase

This comment has been minimized.

Copy link

@futoase futoase commented Dec 29, 2019

Thanks! 🙏

@vaibhavk69

This comment has been minimized.

Copy link

@vaibhavk69 vaibhavk69 commented Jan 12, 2020

i have this produced this raw data and now i only want latitude and longitude pairs, so can anybody help me...........

[11:40 PM, 1/12/2020] Vaibhav Kaushik: {"Response": {"MetaInfo": {"Timestamp": "2020-01-12T18:02:12.167+0000"}, "View": [{"Result": [{"Location": {"Address": {"AdditionalData": [{"key": "CountryName", "value": "India"}, {"key": "StateName", "value": "Uttar Pradesh"}, {"key": "CountyName", "value": "Gautam Buddha Nagar"}], "City": "Noida", "Country": "IND", "County": "Gautam Buddha Nagar", "District": "Sector 125", "Label": "Amity Road, Sector 125, Noida 201303, India", "PostalCode": "201303", "State": "UP", "Street": "Amity Road"}, "DisplayPosition": {"Latitude": 28.54712, "Longitude": 77.33482}, "LocationId": "NT_8RywhVafbhYYOKQDacPkWA", "LocationType": "point", "MapView": {"BottomRight": {"Latitude": 28.54405, "Longitude": 77.33731}, "TopLeft": {"Latitude": 28.5493, "Longitude": 77.33237}}, "NavigationPosition": [{"Latitude": 28.54712, "Longitude": 77.33482}]}, "MatchLevel": "street", "MatchQuality": {"City": 1.0, "Country": 1.0, "District": 1.0, "State": 1.0, "Street": [1.0]}, "Relevance": 1.0}], "ViewId": 0, "_type": "SearchResultsViewType"}]}}

@Palash90

This comment has been minimized.

Copy link

@Palash90 Palash90 commented Jan 12, 2020

Hey Vaibhav, you can use the following -

def find(key, dictionary):
    for k, v in dictionary.items():
        if k == key:
            yield dictionary
        elif isinstance(v, dict):
            for result in find(key, v):
                yield result
        elif isinstance(v, list):
            for d in v:
                if isinstance(d, dict):
                    for result in find(key, d):
                        yield result

print(list(find("Latitude", data)))

I used it on your data and it is showing the following result, hope this helps you -

[{'Latitude': 28.54712, 'Longitude': 77.33482}, {'Latitude': 28.54405, 'Longitude': 77.33731},
        {'Latitude': 28.5493, 'Longitude': 77.33237}, {'Latitude': 28.54712, 'Longitude': 77.33482}]
@chiranjeevijp

This comment has been minimized.

Copy link

@chiranjeevijp chiranjeevijp commented Feb 25, 2020

{'destination_addresses': ['Chennai, Tamil Nadu 600009, India'], 'origin_addresses': ['Chennai, Tamil Nadu 600101, India'], 'rows': [{'elements': [{'distance': {'text': '7.7 mi', 'value': 12327}, 'duration': {'text': '38 mins', 'value': 2285}, 'status': 'OK'}]}], 'status': 'OK'}

Hi Friends, from this JSON i need to extract the value '7.7' which is under the key 'text' under 'distance'. Can any one help in this

@chiranjeevijp

This comment has been minimized.

Copy link

@chiranjeevijp chiranjeevijp commented May 6, 2020

@beeoss

This comment has been minimized.

Copy link

@beeoss beeoss commented Apr 8, 2021

Hello all.. Could anyone help me Python parse and print < "name" , "raw" > (values only), from the JSON below?

{"count":57,"next":null,"previous":null,"results":[{"id":229,"state":"complete","substate":null,"exceptions":[],"name":"Sender Account Number","output_name":null,"field_definition_attributes":{"required":false,"data_type":"Account Number","multiline":false,"routing":false,"supervision_override":null},"transcription":{"raw":"1957-3549-2","normalized":"195735492","source":"machine_transcription","data_deleted":false,"user_transcribed":null},"field_image_url":"/api/v5/image/613cf762-b4bc-46f0-a511-3dc8bb37eae3?start_x=0.3110429607297866&start_y=0.1052441592299208&end_x=0.5696909842243418&end_y=0.16043316955780607","page_id":5}

So for the 'Item' included above, starting at "results": and nesting node to "transcription", the parsed values need to be:

Sender Account Number, 1957-3549-2
(repeating items) 'for'


Much thanks for any assistance!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment