Skip to content

Instantly share code, notes, and snippets.

@a-y-khan

a-y-khan/simple_json_to_df.py

Created February 25, 2020 05:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save a-y-khan/9d3913552624c62fc2ac882c025882ee to your computer and use it in GitHub Desktop.
Save a-y-khan/9d3913552624c62fc2ac882c025882ee to your computer and use it in GitHub Desktop.
Download ZIP
Raw
simple_json_to_df.py
import dask.bag as db
import json
json_bag = db.read_text('test.txt').map(json.loads)
dask_df = json_bag.to_dataframe()
print(dask_df.info())
print(dask_df.head(3))
pandas_df = dask_df.compute()
print(pandas_df.info())
print(pandas_df.head(3))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment