Skip to content

Instantly share code, notes, and snippets.

@shreyasms17
Created May 1, 2021 07:55
Show Gist options
  • Save shreyasms17/afb8b58a37ef24ef7b3d4486bfd23346 to your computer and use it in GitHub Desktop.
Save shreyasms17/afb8b58a37ef24ef7b3d4486bfd23346 to your computer and use it in GitHub Desktop.
AutoFlatten compute
def compute(self):
'''
Description:
This function performs the required computation and gets all the resources
needed for further process of selecting and exploding fields
'''
self.unnest_dict(self.fields_in_json, '')
all_cols_in_explode_cols = set(filter(lambda x: x.startswith(tuple(self.cols_to_explode)), self.all_fields.keys()))
self.rest = set(self.all_fields.keys()).difference(all_cols_in_explode_cols)
self.structure = self.get_structure([f"json{x}" for x in list(self.cols_to_explode)])
self.order = self.extract_order(self.structure)
self.bottom_to_top = self.get_bottom_to_top(self.order, all_cols_in_explode_cols)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment