Skip to content

Instantly share code, notes, and snippets.

@ckingbailey
Created November 12, 2020 02:40
Show Gist options
  • Save ckingbailey/10dcc47676dd33c43789546adda7db94 to your computer and use it in GitHub Desktop.
Save ckingbailey/10dcc47676dd33c43789546adda7db94 to your computer and use it in GitHub Desktop.
I don't get the logic of nested list comprehensions in Python
""" The way nested list comprehensions work looks backwards to me.
I use this pattern a lot to create a collection of DataFrames from stub data
in writing tests for AWS Glue/PySpark ETL jobs.
"""
table_group = [
{
'database': 'ancient_legends_tier_1',
'table_name': 'wizards',
'columns': {
'id': 'int',
'name': 'string',
'birth_epoch': 'int'
},
'rdd': [
[ 1, 'Gandalf', 0 ],
[ 2, 'Morgana', 3 ],
[ 3, 'Galadriel', -10 ]
]
},
{
'database': 'ancient_legends_tier_2',
'table_name': 'epic_battles',
'columns': {
'id': 'int',
'name': 'string',
'wizard_id': 'int',
'duration': 'float',
'epicness': 'float'
},
'rdd': [
[ 1, 'That Time Gandalf Almost Died In The Mines of Moria', 1, 3.2, 10 ],
[ 2, 'Battle of Ealdor', 2, 0.125, 3 ]
]
}
]
table_name = 'epic_battles'
# Intuitively, this seems to me like it should be
# col for col in t['columns'] if t['table_name'] == table_name for t in table_group
headings = [ col for t in table_group if t['table_name'] == table_name for col in t['columns'] ]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment