We will be comparing different compression algorithms, and see which is most efficient, and witch one will be the fastest.
We will compare: zip, tar, gzip, 7z (7zip), and lz4.
Jump to:
The test file is a 244Mb txt file filled with Hello, World! ... (with newlines). Which turns out to be 2033520
lines long, and 256223520 characters.
Each test was ran multiple times, and the compression time was mean of each test.
Here are the results:
| Compression | Speed | Compressed size |
|---|---|---|
| raw | NA | 244Mb |
| 7z | 4.541s | 40Kb |
| zip | 1.200s | 884Kb |
| gzip | 1.100s | 896Kb |
| tar | 1.057s | 908Kb |
| lz4 (slow) | 0.145s | 984Kb |
| lz4 (fast) | 0.141s | 984Kb |
As you can see, the best compression is 7z (by far), and slowest, and the
fastest is lz4, and the least compressed.
Here is each command ran:
| Command | Run |
|---|---|
| zip | zip testfile.txt.zip testfile.txt |
| tar | tar -czf testfile.txt.tar.gz testfile.txt |
| gzip | gzip -k testfile.txt |
| 7z | 7z a testfile.txt.7z testfile.txt |
| lz4 (slow) | lz4 -9 testfile.txt |
| lz4 (fast) | lz4 testfile.txt |
The test directory will be a realistic (pun intented 😄) KSP realision-overhaul and real solar system game directory. It is 5.85Gb and 13064 files, which include some binary files (eg. images, etc...).
gzip was omitted since tar uses gzip compression in this case.
| Compression | Speed | Compressed size |
|---|---|---|
| raw | NA | 5.65Gb |
| 7z | 9m23.179s | 1.8Gb |
| tar | 3m51.134s | 2.3Gb |
| zip | 4m3.726s | 2.4Gb |
| lz4 (slow) | 3m42.326s | 2.5Gb |
| lz4 (fast) | 34.520s | 3.0Gb |
due to the long compressing time, I only ran one test per command.
As expected, 7z has the best compression, but anciently slow. Next would be tar, which
offers good compression vs speed. However, lz4 is still by far the fastest, but much less
compressed (but much better then I was expecting!).
And, for the record, here is the commands ran:
| Command | Run |
|---|---|
| zip | zip -r KSP_1.8_RO.zip KSP_1.8_RO |
| tar | tar -czf KSP_1.8_RO.tar.gz KSP_1.8_RO |
| 7z | 7z a -r KSP_1.8_RO.7z KSP_1.8_RO |
| lz4 (slow) | tar cf - KSP_1.8_RO/ | lz4 -9 > KSP_1.8_RO.tar.lz4 |
| lz4 (fast) | tar cf - KSP_1.8_RO/ | lz4 > KSP_1.8_RO.tar.lz4 |
NOTE: To uncompress the lz4 file, use lz4 -c -d KSP_1.8_RO.tar.lz4 | tar xf - -C ./.
Realistically, if you want to use lz4 to compress a directory, probably just use tar, unless you
need supper fast compression/decompression.