Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Convert any dictionary to a named tuple
from collections import namedtuple
def convert(dictionary):
return namedtuple('GenericDict', dictionary.keys())(**dictionary)
"""
>>> d = dictionary(a=1, b='b', c=[3])
>>> named = convert(d)
>>> named.a == d.a
True
>>> named.b == d.b
True
>>> named.c == d.c
True
"""
@pihentagy

This comment has been minimized.

Copy link

commented Nov 11, 2014

GenericDict will have different fields:

>>> a = convert(dict(c=1,d=2))
>>> b = convert(dict(a=1,b=2))
>>> a
GenericDict(c=1, d=2)
>>> b
GenericDict(a=1, b=2)
@sr-murthy

This comment has been minimized.

Copy link

commented Feb 13, 2015

This is nice, I have to convert a JSON object to a namedtuple via a dict. Could you possibly explain the syntax in this statement return namedtuple('GenericDict', dictionary.keys())(**dictionary), particular the bit with (** <namedtuple name> )?

@ghost

This comment has been minimized.

Copy link

commented Jul 7, 2015

I can.

The function namedtuple() returns a class. Passing **somedict to a class or function as an argument expands the dict into key=value pairs when calling the function. So if I had some function, and some dict like...

def func(kwarg=None):
    return kwarg

derp = { 'kwarg': 39 }

...then I can call func(**derp) and it will return 39. If there are any other key-value pairs in derp, these will expand too, and func will raise an exception.

If you look at namedtuple(), it takes two arguments: a string with the name of the class (which is used by repr like in pihentagy's example), and a list of strings to name the elements. So in the block code above, namedtuple('GenericDict', dictionary.keys()) first returns a class with no particular reference. Passing (**dictionary) right afterwards instantiates the class and returns an instance. If instead we did...

def convert(dictionary):
    NT = namedtuple('GenericDict', dictionary.keys())
    gen_dict = NT(**dictionary)
    return gen_dict

...the same thing would happen. But this way, it might be a little clearer.

@rudifa

This comment has been minimized.

Copy link

commented Jan 6, 2016

A neat idea!
Here is my variation on this theme:

from collections import namedtuple

def MakeNamedTuple(name='NamedTuple', **kwargs):
   """
    Returns a namedtuple instance.
    >>> nt1 = MakeNamedTuple(reclen=1000, numrec=5, samprate=1e6)
    >>> nt1
    NamedTuple(numrec=5, reclen=1000, samprate=1000000.0)
    >>> nt1.reclen
    1000
    >>> sp1 = MakeNamedTuple(name='Setup', reclen=2000, numrec=55, samprate=1.6e6)
    >>> sp1
    Setup(numrec=55, reclen=2000, samprate=1600000.0)
    >>> sp1.samprate
    1600000.0
    """
    dt = dict(**kwargs)
    return namedtuple(name, sorted(dt))(**dt)
@suganthsundar

This comment has been minimized.

Copy link

commented Jan 27, 2016

Thanks. Added code to convert it recursively.

from collections import namedtuple

def convert(dictionary):
    for key, value in dictionary.iteritems():
            if isinstance(value, dict):
                dictionary[key] = convert(value) 
    return namedtuple('GenericDict', dictionary.keys())(**dictionary)
@arisada

This comment has been minimized.

Copy link

commented Feb 2, 2016

I have a variation to propose, in which you're 100% sure it will continue to work if new members are added to the dict (json API evolving):

args='a1 a2 a3'
d={"a1":1,"a2":2,"a3":3}
X=namedtuple("Blah",args)
X._make(d[i] for i in args.split(" "))

Blah(a1=1, a2=2, a3=3)

@purpleP

This comment has been minimized.

Copy link

commented Jun 16, 2016

I'd like to point out the performance of this.

For a dictionary of 20 keys the conversion takes 500 us on my machine. Member access is about 3 times slower than of original dict. That's on cpython. On pypy surprisingly conversion takes a about 1 ms (2 times slower than cpython) and access is a little bit faster in nametuple. The process of creating a namedtuple is taking most of the time of convertion.

@asfaltboy

This comment has been minimized.

Copy link

commented Aug 18, 2016

For a quick and dirty dict keys to object attrs conversion, I use mock:

import mock

d = dict(a=1, b='b')
o = mock.Mock(**d)
assert d['a'] == o.a
assert d['b'] == o.b
@twslankard

This comment has been minimized.

Copy link

commented Sep 7, 2016

This approach is likely to be error prone because python dictionaries aren't ordered but namedtuple is ordered. You could get different positions in the tuple for different fields from run to run.

For example, this program (when using Python 3.4.X) occasionally throws an error. Run it a few times to reproduce it.

import collections
Point = collections.namedtuple('Point', {'x':0, 'y':0})
p = Point(11, y=22)     # instantiate with positional or keyword arguments
p[0] + p[1]             # indexable like the plain tuple (11, 22)

The error looks like this:

Traceback (most recent call last):
  File "tupletest.py", line 5, in <module>
    p = Point(11, y=22)     # instantiate with positional or keyword arguments
TypeError: __new__() got multiple values for argument 'y'
@wtrevino

This comment has been minimized.

Copy link

commented Nov 2, 2016

I added support for list values to suganthsundar's solution:

def convert(dictionary):
    for key, value in dictionary.items():
            if isinstance(value, dict):
                dictionary[key] = convert(value)
            if isinstance(value, list):
                dictionary[key] = [convert(i) for i in value]
    return namedtuple('GenericDict', dictionary.keys())(**dictionary)
@JinghongHuang

This comment has been minimized.

Copy link

commented Apr 11, 2017

Above code breaks if the item of the list is not a dictionary. Modifying the code as following, which should work for most of the cases:

from collections import namedtuple
def convert(obj):
    if isinstance(obj, dict):
        for key, value in obj.iteritems():
            obj[key] = convert(value) 
        return namedtuple('GenericDict', obj.keys())(**obj)
    elif isinstance(obj, list):
        return [convert(item) for item in obj]
    else:
        return obj

@liiight

This comment has been minimized.

Copy link

commented Oct 25, 2017

Thanks for this!

@junjizhi

This comment has been minimized.

Copy link

commented Aug 2, 2018

@JinghongHuang The snippet you posted above has bad performance because it creates many GenericDict classes recursively and in python these classes are never garbage recycled so it will memory leaks!

@haydenflinner

This comment has been minimized.

Copy link

commented Oct 12, 2018

Anyone interested in this should also check out the dotmap package / alternatives to that package

@malcolmgreaves

This comment has been minimized.

Copy link

commented Dec 11, 2018

I wrote up this [1] implementation a little while ago. Stumbled upon this thread. From the namedtuple_fmt module, the serialize function accepts a dictionary (e.g. from a json.load call) and will convert to an instance of the given namedtuple-deriving type. Likewise, deserialize will convert any namedtuple-deriving type into a dictionary-l object. These functions also work on any list-like type: List and Tuple are process-able.

E.g.

import json
from typing import Sequence

from namedtuple_fmt import serialize, deserialize

X = NamedTuple('X', [('msg',str)])

json_str="""{"msg": "This is the first message"}"""
first_msg = deserialize(json.loads(json_str), X) 
print(first_msg.msg)
print(deserialize(serialize(first_msg)) == X("This is the first message"))
print(deserialize(json.loads(json.dumps(serialize(first_msg)))) == X("This is the first message"))

json_str="""[{"msg": "This is the first message"},{"msg": "This is the 2nd message"}]"""
messages = deserialize(json.loads(json_str), Sequence[X])
print(f"{len(messages)} messages")
print('\n'.join(map(lambda x: x.msg, messages))

Implementation note: There are explicit type checks for the Sequence types. It's important to not mess-up when it comes to handling str and tuple. A first draft of this idea incorrectly did for _ in X when trying to "test" if something was list-like. This idea, unfortunately, will iterate over characters in a str or elements in a tuple. We want the tuple-iterating part, but not when it comes to a NamedTuple (or namedtuple)!

[1] https://gist.github.com/malcolmgreaves/d71ae1f09075812e54d8ec54a5613616

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.