Skip to content

Instantly share code, notes, and snippets.

@lockefox
Created August 22, 2017 19:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lockefox/fb387707e9f8f9b9e421ef10660fc7a4 to your computer and use it in GitHub Desktop.
Save lockefox/fb387707e9f8f9b9e421ef10660fc7a4 to your computer and use it in GitHub Desktop.
data_columns = ['killID', 'solarSystemID', 'killTime', 'moonID']
pivot_column = 'items'
index_column = 'killID'
## Make source DataFrame ##
raw_df = pd.DataFrame(raw_data)
## Rotate list-column into its own frame ##
pivot_df = pd.DataFrame(list(raw_df[pivot_column]))
pivot_df = pd.concat(raw_df[index_column, pivot_df], axis=1) #add index to frame
## Melt down data column into a single column ##
pivot_df = pd.melt(pivot_df, id_vars=[index_column]) #compress arbitrary-len data into one col
pivot_df.dropna(axis=0, how='any', inplace=True) #remove blank rows
## Apply dict-pivot to data ##
value_df = pd.DataFrame(list(pivot_df['value']))
value_df = pd.concat([pivot_df[index_column], value_df], axis=1) #add index to frame
## Merge source/pivot together ##
result_df = raw_df.join(value_df, index_column, lsuffix='_')
## Tidy DF ##
data_columns.extend(list(value_df.columns.values))
drop_columns = list(set(result_df.columns.values) - set(data_columns))
result_df = result_df.drop(drop_columns, 1)
flag killID killTime moonID qtyDestroyed qtyDropped singleton solarSystemID typeID
89 64196865 2017-08-21 04:12:23 0 1 0 0 31002561 9942
89 64196865 2017-08-21 04:12:23 0 1 0 0 31002561 9899
89 64196865 2017-08-21 04:12:23 0 1 0 0 31002561 9941
12 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 2605
28 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 25812
93 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 31153
11 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 2605
29 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 11578
20 64196863 2017-08-21 04:12:13 0 0 1 0 31002561 3831
92 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 31153
21 64196863 2017-08-21 04:12:13 0 0 1 0 31002561 2333
27 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 25812
134 64196863 2017-08-21 04:12:13 0 0 416 0 31002561 30376
13 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 11640
134 64196863 2017-08-21 04:12:13 0 1996 0 0 31002561 30375
14 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 11640
19 64196863 2017-08-21 04:12:13 0 1 0 0 31002561 35658
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment