Skip to content

Instantly share code, notes, and snippets.

@gautierdag

gautierdag/numpy_compress.py

Last active July 15, 2020 10:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gautierdag/84aa55297babc5fb211bac67e08b00f7 to your computer and use it in GitHub Desktop.
Save gautierdag/84aa55297babc5fb211bac67e08b00f7 to your computer and use it in GitHub Desktop.
Download ZIP
Numpy Optimized Compress
Raw
numpy_compress.py
import numpy as np
#cast as object to be able to set np.nan and handle different types
arr = df[cols].values.astype(object)
# build shifted array
shifted = np.roll(arr, 1)
shifted[0] = np.nan
# choose indexes based on shift comparaison
indexes = np.where(~(arr == shifted).all(axis=1))[0]
compressed_df = df.loc[indexes].reset_index(drop=True)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment