Skip to content

Instantly share code, notes, and snippets.

@andreagrioni

andreagrioni/write_h5.py

Created May 14, 2021 21:56
Show Gist options
  • Save andreagrioni/af64b822b01affd3170c434fba4c0a8a to your computer and use it in GitHub Desktop.
Save andreagrioni/af64b822b01affd3170c434fba4c0a8a to your computer and use it in GitHub Desktop.
Download ZIP
write pandas dataframes as h5
Raw
write_h5.py
import h5py
# create HDF5 file
with h5py.File(PROCESSPATH.joinpath("final_df.h5"), 'w') as hf:
dset_x_train = hf.create_dataset(
'x_train', data=X_train, shape=X_train.shape, compression='gzip', chunks=True)
dset_y_train = hf.create_dataset(
'y_train', data=y_train, shape=y_train.shape, compression='gzip', chunks=True)
dset_x_test = hf.create_dataset(
'x_test', data=X_test, shape=X_test.shape, compression='gzip', chunks=True)
dset_y_test = hf.create_dataset(
'y_test', data=y_test, shape=y_test.shape, compression='gzip', chunks=True)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment