Skip to content

Instantly share code, notes, and snippets.

@kozo496

kozo496/convert_dask2pandas.py Secret

Last active June 12, 2022 11:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kozo496/3d99170df2824ba9b273ceab4db8ec29 to your computer and use it in GitHub Desktop.
Save kozo496/3d99170df2824ba9b273ceab4db8ec29 to your computer and use it in GitHub Desktop.
Download ZIP
convert dask dataframe to pandas.dataframe (ref: https://coiled.io/blog/converting-a-dask-dataframe-to-a-pandas-dataframe/)
Raw
convert_dask2pandas.py
# https://coiled.io/blog/converting-a-dask-dataframe-to-a-pandas-dataframe/
import dask.dataframe as dd
import pandas as pd
df = pd.DataFrame(
{"nums": [1, 2, 3, 4, 5, 6], "letters": ["a", "b", "c", "d", "e", "f"]}
)
ddf = dd.from_pandas(df, npartitions=2)
# dask.dataframeをpandas.dataframeに変換
pandas_df = ddf.compute()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment