Skip to content

Instantly share code, notes, and snippets.

@btheu
Created March 4, 2024 19:40
Show Gist options
  • Save btheu/a5ca794016a26964243861f1f2a6ff00 to your computer and use it in GitHub Desktop.
Save btheu/a5ca794016a26964243861f1f2a6ff00 to your computer and use it in GitHub Desktop.
docker run --rm -it python:3.9-slim bash
pip install pandas pyarrow
python
import pandas as pd
import numpy as np
import json
dict =[{ 'a':1.1, 'b':1.2 },
{ 'a':2.1, 'b':2.2 },
{ 'a':3.1, 'b':np.nan }
]
df = pd.DataFrame(dict)
df = pd.DataFrame([])
json_str = '{"a":1.1, "b":1.2, "c": 1.3 }'
data = [json.loads(json_str)]
df = pd.concat([df,pd.DataFrame(data)])
df = pd.concat([df,pd.DataFrame( [json.loads('{"a":1.1, "b":1.2, "c": NaN }')])])
print(df)
df.to_parquet('out-json.parquet')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment