Skip to content

Instantly share code, notes, and snippets.

@ramcandrews
Last active September 27, 2019 06:14
Show Gist options
  • Save ramcandrews/41ab9ddce3be197139b6a5fbc6ebd5cb to your computer and use it in GitHub Desktop.
Save ramcandrews/41ab9ddce3be197139b6a5fbc6ebd5cb to your computer and use it in GitHub Desktop.
Normally you can't remove items from a dictionary while you are iterating over it. But if you convert it to a list first, there are no problems.
origDict = {
'First name': 'Ryan',
'Last name': 'M',
'Subject': 'AI',
'task': 'Cleaning Data'
}
removedItem = origDict.pop('Last name') # this is normal usage of pop()
print(origDict)
print('value = ' + removedItem)
# now lets loop and pop()
newDict = {} # create a new empty dictionary for us to dump our items() into
# create a list comprehension to build a list from the original dictionary that holds the data we want to process.
tmp = [(k, v) for k, v in origDict.items()]
# now instead of looping through the dictionary,
# we loop through the list
# and pop items out of the original dictionary as we add items to the new dictionary!
for k, v in tmp:
# you can process the data here, before you put it in another dictionary.
newDict[k] = origDict.pop(k, v)
# in Python, pop() removes an item from a list/dictionary and returns it. In this case we are stuffing it into a new
# dictionary right away.
# Or you can put the comprehension in the loop!
for k, v in [(k, v) for k, v in origDict.items()]:
newDict[k] = origDict.pop(k, v)
print(origDict)
print(newDict)
@ramcandrews
Copy link
Author

I wanted to remove items from a dictionary while iterating over it because I was cleaning data and needed to conditionally process data line by line. The problem was that there was no straightforward way to loop through the lines without processing some lines more than once.

Also, by using a list comprehension you end up only using a few lines of code to complete a pretty complex task.

@rodolfoams
Copy link

A different solution would be the following:

newDict = {k: origDict.pop(k) for k in list(origDict.keys())}

@ramcandrews
Copy link
Author

A different solution would be the following:

newDict = {k: origDict.pop(k) for k in list(origDict.keys())}

Thanks!
How can I process the values while iterating over that?

@rodolfoams
Copy link

One way could be the following:

def my_filter(value):
    # Check here if the item should be removed
    pass

def process(k, value):
    # Define here how to process the item
    pass

newDict = {k: process(k, origDict.pop(k)) for k in list(origDict.keys()) if my_filter(origDict[k])}

@ramcandrews
Copy link
Author

One way could be the following:

def my_filter(value):
    # Check here if the item should be removed
    pass

def process(k, value):
    # Define here how to process the item
    pass

newDict = {k: process(k, origDict.pop(k)) for k in list(origDict.keys()) if my_filter(origDict[k])}

Thanks, this is really useful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment