Skip to content

Instantly share code, notes, and snippets.

@stuarteberg
Last active June 15, 2020 14:23
Show Gist options
  • Save stuarteberg/85074550fc0b39f653b7b789c169f337 to your computer and use it in GitHub Desktop.
Save stuarteberg/85074550fc0b39f653b7b789c169f337 to your computer and use it in GitHub Desktop.
Quick test of compressing random data with/without shuffling
In [20]: import numpy as np
...: import lz4.frame
...:
...: a = np.random.randint(2**16, size=1_000_000, dtype=int)
...: buf = lz4.frame.compress(a)
...: print("Compressed size without shuffling:", len(buf))
...:
...: shuffled = a.view(np.uint8).reshape(-1, 8).transpose().copy(order='C')
...: buf2 = lz4.frame.compress(shuffled)
...: print("Compressed size with shuffling: ", len(buf2))
...:
Compressed size without shuffling: 4077762
Compressed size with shuffling: 2025044
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment