Skip to content

Instantly share code, notes, and snippets.

@TheTechRobo

TheTechRobo/BENCHMARKS

Last active February 11, 2022 14:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save TheTechRobo/091acf60f80d007557ad35821fdb3a6d to your computer and use it in GitHub Desktop.
Save TheTechRobo/091acf60f80d007557ad35821fdb3a6d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
BENCHMARKS
=====================
#lrz/zstd benchmarks#
=====================
Source File
-----------
A 700-ish MB jsonl file
lrzip (LZO)
-----------
tweets.jsonl - Compression Ratio: 11.289. Average Compression Speed: 35.850MB/s.
Total time: 00:00:19.51
lrzip (Default (LZMA))
----------------------
tweets.jsonl - Compression Ratio: 20.759. Average Compression Speed: 19.378MB/s.
Total time: 00:00:36.91
zstd
----
tweets.jsonl : 7.31% (752551172 => 55039290 bytes, tweets.jsonl.zst)
zstd -z tweets.jsonl 1.10s user 0.12s system 117% cpu 1.035 total
RESULT
======
zstd has a slightly better compression ratio when compared to lrzip/LZO. lrzip/LZO took 19x the time zstd did!
lrzip/LZMA had a much better compression ratio, but took a whopping 37 times the amount of time zstd did!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment