Created
April 12, 2024 14:45
-
-
Save cswinter/853e53f58c5745bc84db3ddcbe6abf35 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Results of comparing pco and xor compression at https://github.com/cswinter/LocustDB/commit/46edbce88f2aceb3f5c12da2fcc123486390ff58 | |
| $ cargo run --release --example gorilla_time | |
| Compression ratio of 3.17 for trace/rollout/forward/action_heads (max_regret=100) | |
| Compression ratio of 5.68 for actions/act_right_up (max_regret=100) | |
| Compression ratio of 2.36 for episode_length.mean (max_regret=100) | |
| Compression ratio of 11.95 for reward.max (max_regret=100) | |
| Compression ratio of 2.37 for gradnorm (max_regret=100) | |
| Compression ratio of 13.52 for sps (max_regret=100) | |
| Compression ratio of 2.20 for approxkl (max_regret=100) | |
| Compression ratio of 2.22 for policy_loss (max_regret=100) | |
| Compression ratio of 3.05 for trace/update.optimize.broadcast_advantages (max_regret=100) | |
| Compression ratio of 6.98 for step (max_regret=100) | |
| Compression ratio of 1.00 for evil_numbers (max_regret=100) | |
| Compression ratio of 1.80 for timestamp (max_regret=100) | |
| $ cargo run --release --example gorilla_time -- --pco | |
| Compression ratio of 3.55 for trace/rollout/forward/action_heads (pcodec) | |
| Compression ratio of 6.90 for actions/act_right_up (pcodec) | |
| Compression ratio of 2.49 for episode_length.mean (pcodec) | |
| Compression ratio of 20.21 for reward.max (pcodec) | |
| Compression ratio of 2.36 for gradnorm (pcodec) | |
| Compression ratio of 11.95 for sps (pcodec) | |
| Compression ratio of 2.17 for approxkl (pcodec) | |
| Compression ratio of 2.30 for policy_loss (pcodec) | |
| Compression ratio of 3.76 for trace/update.optimize.broadcast_advantages (pcodec) | |
| Compression ratio of 28.82 for step (pcodec) | |
| Compression ratio of 0.89 for evil_numbers (pcodec) | |
| Compression ratio of 2.84 for timestamp (pcodec) | |
| xor pco data | |
| 3.17 3.55 trace/rollout/forward/action_heads | |
| 5.68 6.90 actions/act_right_up | |
| 2.36 2.49 episode_length.mean | |
| 11.95 20.21 reward.max | |
| 2.37 2.36 gradnorm | |
| 13.52 11.95 sps | |
| 2.20 2.17 approxkl | |
| 2.22 2.30 policy_loss | |
| 3.05 3.76 trace/update.optimize.broadcast_advantages | |
| 6.98 28.82 step | |
| 1.00 0.89 evil_numbers | |
| 1.80 2.84 (pco) timestamp |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Very cool! I took a look at
evil_numbersandspsdatasets, and pco underperformed there due to the tiny data sizes (sps was 136 bytes compressed). Looks like pco did as well or better in the other cases, as I'd expect.