Skip to content

Instantly share code, notes, and snippets.

@Artefact2

Artefact2/00considerations.md

Last active September 1, 2021 02:59
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save Artefact2/eb4b47d8c3310ad537547c2c445446f5 to your computer and use it in GitHub Desktop.
Save Artefact2/eb4b47d8c3310ad537547c2c445446f5 to your computer and use it in GitHub Desktop.
Download ZIP
Borg compression benchmark/comparison
Raw
00considerations.md

Considerations

  • Tests were run in single-user mode, one at a time.
  • Data was read from and written to tmpfs.
  • borg-1.1rc2 seems about 10% slower than borg-1.0.11 (many possible reasons, borg-1.1rc2 binary comes from GitHub but 1.0.11 from the Arch repos)
  • Still making sense of the auto,X,Y compression modes
Raw
01dataset.md

Dataset 1: linux 4.13 mainline sources (many small files, good compressibility)

Benchmark date: 2017-09-09
Borg version: borg-linux64 1.1.0rc2
Python version: Python 3.6.2
Kernel version: Linux 4.12.8-2-ARCH
CPU model: AMD FX(tm)-8150 Eight-Core Processor

Borg init command: %s init --error -e repokey %s
Borg create command: %s create --error -C %s %s::foo %s
Dataset path: /tmp/borg-dataset (tmpfs)
Repository path: /tmp/borg-repo (tmpfs)

Averaging algos over 1 run(s)

           algo |    cputime |  reposize |   bytes/s | timeratio | compratio
----------------+------------+-----------+-----------+-----------+----------
           none |      31.10 |  792.654M |   25.487M |      1.00 |     1.000
            lz4 |      29.54 |  311.050M |   26.833M |      0.95 |     0.392
         zlib,0 |      33.00 |  793.244M |   24.020M |      1.06 |     1.001
         zlib,1 |      36.82 |  228.762M |   21.528M |      1.18 |     0.289
         zlib,2 |      37.74 |  221.245M |   21.003M |      1.21 |     0.279
         zlib,3 |      39.05 |  216.240M |   20.298M |      1.26 |     0.273
         zlib,4 |      41.51 |  205.451M |   19.095M |      1.33 |     0.259
         zlib,5 |      44.57 |  199.655M |   17.784M |      1.43 |     0.252
         zlib,6 |      50.48 |  197.501M |   15.702M |      1.62 |     0.249
         zlib,7 |      55.10 |  196.702M |   14.386M |      1.77 |     0.248
         zlib,8 |      66.13 |  196.170M |   11.986M |      2.13 |     0.247
         zlib,9 |      66.77 |  196.137M |   11.871M |      2.15 |     0.247
         lzma,0 |      79.57 |  201.884M | 9961.717K |      2.56 |     0.255
         lzma,1 |      96.85 |  195.178M | 8184.345K |      3.11 |     0.246
         lzma,2 |     114.55 |  192.754M | 6919.719K |      3.68 |     0.243
         lzma,3 |     154.87 |  191.959M | 5118.188K |      4.98 |     0.242
         lzma,4 |     307.70 |  189.075M | 2576.060K |      9.89 |     0.239
         lzma,5 |     487.64 |  185.676M | 1625.489K |     15.68 |     0.234
         lzma,6 |     543.37 |  184.078M | 1458.773K |     17.47 |     0.232
         lzma,7 |     764.68 |  184.058M | 1036.582K |     24.59 |     0.232
         lzma,8 |    1218.62 |  184.082M |  650.452K |     39.18 |     0.232
         lzma,9 |    1218.07 |  184.095M |  650.745K |     39.17 |     0.232
       auto,lz4 |      32.19 |  311.050M |   24.624M |      1.04 |     0.392
    auto,zlib,0 |      36.56 |  793.240M |   21.681M |      1.18 |     1.001
    auto,zlib,1 |      39.51 |  228.762M |   20.062M |      1.27 |     0.289
    auto,zlib,2 |      41.02 |  221.250M |   19.324M |      1.32 |     0.279
    auto,zlib,3 |      41.67 |  216.248M |   19.022M |      1.34 |     0.273
    auto,zlib,4 |      44.01 |  205.447M |   18.011M |      1.42 |     0.259
    auto,zlib,5 |      47.52 |  199.660M |   16.680M |      1.53 |     0.252
    auto,zlib,6 |      53.45 |  197.497M |   14.830M |      1.72 |     0.249
    auto,zlib,7 |      57.71 |  196.706M |   13.735M |      1.86 |     0.248
    auto,zlib,8 |      67.28 |  196.162M |   11.781M |      2.16 |     0.247
    auto,zlib,9 |      69.64 |  196.141M |   11.382M |      2.24 |     0.247
    auto,lzma,0 |      82.29 |  201.830M | 9632.444K |      2.65 |     0.255
    auto,lzma,1 |     100.03 |  195.105M | 7924.160K |      3.22 |     0.246
    auto,lzma,2 |     116.15 |  192.668M | 6824.397K |      3.73 |     0.243
    auto,lzma,3 |     159.19 |  192.012M | 4979.294K |      5.12 |     0.242
    auto,lzma,4 |     314.17 |  189.080M | 2523.009K |     10.10 |     0.239
    auto,lzma,5 |     486.67 |  185.709M | 1628.729K |     15.65 |     0.234
    auto,lzma,6 |     542.93 |  184.013M | 1459.955K |     17.46 |     0.232
    auto,lzma,7 |     763.56 |  184.054M | 1038.102K |     24.55 |     0.232
    auto,lzma,8 |    1207.50 |  184.029M |  656.442K |     38.83 |     0.232
    auto,lzma,9 |    1205.69 |  184.001M |  657.427K |     38.77 |     0.232
Raw
02dataset.md

Dataset 2: postgres dump (large text files, good compressibility)

Benchmark date: 2017-09-10
Borg version: borg-linux64 1.1.0rc2
Python version: Python 3.6.2
Kernel version: Linux 4.12.8-2-ARCH
CPU model: AMD FX(tm)-8150 Eight-Core Processor

Borg init command: %s init --error -e repokey %s
Borg create command: %s create --error -C %s %s::foo %s
Dataset path: /tmp/borg-dataset (tmpfs)
Repository path: /tmp/borg-repo (tmpfs)

Averaging algos over 1 run(s)

           algo |    cputime |  reposize |   bytes/s | timeratio | compratio
----------------+------------+-----------+-----------+-----------+----------
           none |      12.06 |  682.570M |   56.598M |      1.00 |     1.000
            lz4 |      10.04 |  222.540M |   67.985M |      0.83 |     0.326
         zlib,0 |      14.91 |  682.672M |   45.779M |      1.24 |     1.000
         zlib,1 |      15.61 |  154.948M |   43.726M |      1.29 |     0.227
         zlib,2 |      16.50 |  150.839M |   41.368M |      1.37 |     0.221
         zlib,3 |      18.30 |  147.550M |   37.299M |      1.52 |     0.216
         zlib,4 |      21.61 |  136.143M |   31.586M |      1.79 |     0.199
         zlib,5 |      25.56 |  128.610M |   26.705M |      2.12 |     0.188
         zlib,6 |      31.23 |  126.640M |   21.856M |      2.59 |     0.186
         zlib,7 |      36.63 |  125.997M |   18.634M |      3.04 |     0.185
         zlib,8 |      51.20 |  124.961M |   13.331M |      4.25 |     0.183
         zlib,9 |      53.49 |  124.903M |   12.761M |      4.44 |     0.183
         lzma,0 |      43.15 |  125.125M |   15.819M |      3.58 |     0.183
         lzma,1 |      47.43 |  115.737M |   14.391M |      3.93 |     0.170
         lzma,2 |      59.86 |  111.706M |   11.403M |      4.96 |     0.164
         lzma,3 |      77.78 |  109.818M | 8775.645K |      6.45 |     0.161
         lzma,4 |     129.02 |   98.345M | 5290.417K |     10.70 |     0.144
         lzma,5 |     174.62 |   94.564M | 3908.886K |     14.48 |     0.139
         lzma,6 |     320.48 |   89.686M | 2129.835K |     26.57 |     0.131
         lzma,7 |     322.40 |   89.711M | 2117.151K |     26.73 |     0.131
         lzma,8 |     326.85 |   89.489M | 2088.327K |     27.10 |     0.131
         lzma,9 |     329.77 |   89.518M | 2069.835K |     27.34 |     0.131
       auto,lz4 |      11.78 |  222.540M |   57.943M |      0.98 |     0.326
    auto,zlib,0 |      16.60 |  682.672M |   41.119M |      1.38 |     1.000
    auto,zlib,1 |      17.35 |  154.952M |   39.341M |      1.44 |     0.227
    auto,zlib,2 |      18.13 |  150.847M |   37.649M |      1.50 |     0.221
    auto,zlib,3 |      20.02 |  147.522M |   34.094M |      1.66 |     0.216
    auto,zlib,4 |      23.53 |  136.163M |   29.008M |      1.95 |     0.199
    auto,zlib,5 |      26.54 |  128.618M |   25.719M |      2.20 |     0.188
    auto,zlib,6 |      34.28 |  126.648M |   19.912M |      2.84 |     0.186
    auto,zlib,7 |      37.69 |  125.989M |   18.110M |      3.13 |     0.185
    auto,zlib,8 |      53.51 |  124.953M |   12.756M |      4.44 |     0.183
    auto,zlib,9 |      55.94 |  124.891M |   12.202M |      4.64 |     0.183
    auto,lzma,0 |      45.07 |  125.088M |   15.145M |      3.74 |     0.183
    auto,lzma,1 |      49.24 |  115.806M |   13.862M |      4.08 |     0.170
    auto,lzma,2 |      61.87 |  111.907M |   11.032M |      5.13 |     0.164
    auto,lzma,3 |      78.91 |  110.121M | 8649.977K |      6.54 |     0.161
    auto,lzma,4 |     135.67 |   97.944M | 5031.102K |     11.25 |     0.143
    auto,lzma,5 |     173.32 |   94.376M | 3938.205K |     14.37 |     0.138
    auto,lzma,6 |     321.33 |   89.723M | 2124.201K |     26.64 |     0.131
    auto,lzma,7 |     324.67 |   89.477M | 2102.349K |     26.92 |     0.131
    auto,lzma,8 |     325.54 |   89.559M | 2096.730K |     26.99 |     0.131
    auto,lzma,9 |     327.80 |   89.825M | 2082.274K |     27.18 |     0.132
Raw
03dataset.md

Dataset 3: RAW photo collection (large files, poor compressibility)

Benchmark date: 2017-09-10
Borg version: borg-linux64 1.1.0rc2
Python version: Python 3.6.2
Kernel version: Linux 4.12.8-2-ARCH
CPU model: AMD FX(tm)-8150 Eight-Core Processor

Borg init command: %s init --error -e repokey %s
Borg create command: %s create --error -C %s %s::foo %s
Dataset path: /tmp/borg-dataset (tmpfs)
Repository path: /tmp/borg-repo (tmpfs)

Averaging algos over 1 run(s)

           algo |    cputime |  reposize |   bytes/s | timeratio | compratio
----------------+------------+-----------+-----------+-----------+----------
           none |      10.08 |  552.243M |   54.786M |      1.00 |     1.000
            lz4 |      10.51 |  553.198M |   52.545M |      1.04 |     1.002
         zlib,0 |      12.31 |  552.329M |   44.861M |      1.22 |     1.000
         zlib,1 |      31.31 |  541.327M |   17.638M |      3.11 |     0.980
         zlib,2 |      30.76 |  541.123M |   17.953M |      3.05 |     0.980
         zlib,3 |      32.22 |  541.028M |   17.140M |      3.20 |     0.980
         zlib,4 |      32.63 |  541.639M |   16.924M |      3.24 |     0.981
         zlib,5 |      33.09 |  541.610M |   16.689M |      3.28 |     0.981
         zlib,6 |      34.04 |  541.594M |   16.223M |      3.38 |     0.981
         zlib,7 |      32.58 |  541.594M |   16.950M |      3.23 |     0.981
         zlib,8 |      33.74 |  541.594M |   16.368M |      3.35 |     0.981
         zlib,9 |      32.60 |  541.598M |   16.940M |      3.23 |     0.981
         lzma,0 |     114.12 |  539.759M | 4839.144K |     11.32 |     0.977
         lzma,1 |     128.01 |  539.451M | 4314.062K |     12.70 |     0.977
         lzma,2 |     139.48 |  539.443M | 3959.300K |     13.84 |     0.977
         lzma,3 |     144.05 |  539.423M | 3833.691K |     14.29 |     0.977
         lzma,4 |     152.77 |  536.760M | 3614.866K |     15.16 |     0.972
         lzma,5 |     150.73 |  536.711M | 3663.790K |     14.95 |     0.972
         lzma,6 |     150.52 |  536.781M | 3668.902K |     14.93 |     0.972
         lzma,7 |     148.66 |  536.748M | 3714.806K |     14.75 |     0.972
         lzma,8 |     149.84 |  536.805M | 3685.552K |     14.87 |     0.972
         lzma,9 |     149.07 |  536.736M | 3704.589K |     14.79 |     0.972
       auto,lz4 |      10.69 |  551.404M |   51.660M |      1.06 |     0.998
    auto,zlib,0 |      10.99 |  552.255M |   50.250M |      1.09 |     1.000
    auto,zlib,1 |      13.00 |  546.533M |   42.480M |      1.29 |     0.990
    auto,zlib,2 |      13.37 |  545.362M |   41.305M |      1.33 |     0.988
    auto,zlib,3 |      13.57 |  545.980M |   40.696M |      1.35 |     0.989
    auto,zlib,4 |      14.26 |  545.980M |   38.727M |      1.41 |     0.989
    auto,zlib,5 |      15.30 |  544.649M |   36.094M |      1.52 |     0.986
    auto,zlib,6 |      13.90 |  546.619M |   39.730M |      1.38 |     0.990
    auto,zlib,7 |      14.00 |  546.296M |   39.446M |      1.39 |     0.989
    auto,zlib,8 |      13.67 |  546.947M |   40.398M |      1.36 |     0.990
    auto,zlib,9 |      14.01 |  546.161M |   39.418M |      1.39 |     0.989
    auto,lzma,0 |      21.26 |  544.084M |   25.976M |      2.11 |     0.985
    auto,lzma,1 |      23.84 |  544.461M |   23.165M |      2.37 |     0.986
    auto,lzma,2 |      24.53 |  544.211M |   22.513M |      2.43 |     0.985
    auto,lzma,3 |      25.64 |  544.330M |   21.538M |      2.54 |     0.986
    auto,lzma,4 |      31.46 |  542.163M |   17.554M |      3.12 |     0.982
    auto,lzma,5 |      28.74 |  542.806M |   19.215M |      2.85 |     0.983
    auto,lzma,6 |      27.20 |  543.543M |   20.303M |      2.70 |     0.984
    auto,lzma,7 |      25.30 |  544.756M |   21.828M |      2.51 |     0.986
    auto,lzma,8 |      26.69 |  543.953M |   20.691M |      2.65 |     0.985
    auto,lzma,9 |      27.53 |  542.675M |   20.060M |      2.73 |     0.983
Raw
98borg-compressbench.php
#!/usr/bin/env php
<?php
/* Author: Romain "Artefact2" Dal Maso <artefact2@gmail.com> */
/* This program is free software. It comes without any warranty, to the
* extent permitted by applicable law. You can redistribute it and/or
* modify it under the terms of the Do What The Fuck You Want To Public
* License, Version 2, as published by Sam Hocevar. See
* http://sam.zoy.org/wtfpl/COPYING for more details. */
const BORG_BINARY = '/home/romain/Downloads/borg-linux64';
const DATASET_DIR = '/tmp/borg-dataset';
const BORG_REPO = '/tmp/borg-repo';
const STATE_FILE = '/tmp/borg-compressbench.state';
const INIT_COMMAND = '%s init --error -e repokey %s';
const CREATE_COMMAND = '%s create --error -C %s %s::foo %s';
const RUNS = 1;
$compressions = [ 'lz4' ];
foreach([ 'zlib', 'lzma' ] as $algo) {
foreach(range(0, 9) as $level) {
$compressions[] = $algo.','.$level;
}
}
foreach($compressions as $comp) {
$compressions[] = 'auto,'.$comp;
}
array_unshift($compressions, 'none');
printf("Benchmark date: %s\n", gmdate('Y-m-d'));
printf("Borg version: %s", shell_exec(sprintf('%s --version', escapeshellarg(BORG_BINARY))));
printf("Python version: %s", shell_exec('python --version'));
printf("Kernel version: %s", shell_exec('uname -sr'));
printf("CPU model: %s", shell_exec('grep "model name" /proc/cpuinfo | head -n1 | cut -d\' \' -f3-'));
printf("\nBorg init command: %s\n", INIT_COMMAND);
printf("Borg create command: %s\n", CREATE_COMMAND);
printf("Dataset path: %s (%s)\n", DATASET_DIR, find_mountpoint(DATASET_DIR));
printf("Repository path: %s (%s)\n", BORG_REPO, find_mountpoint(BORG_REPO));
printf("\nAveraging algos over %d run(s)\n", RUNS);
putenv('BORG_PASSPHRASE=foobar');
if(file_exists(STATE_FILE)) {
$results = json_decode(file_get_contents(STATE_FILE), true);
fprintf(STDERR, "\nWARNING: resuming progress from state file %s\nDelete file if you want to start over.\n", STATE_FILE);
} else {
$results = [];;
}
const FORMAT = "%15s | %10s | %9s | %9s | %9s | %9s\n";
const IFORMAT = "%15s | %10.2f | %9s | %9s | %9.2f | %9.3f\n";
printf("\n".FORMAT, "algo", "cputime", "reposize", "bytes/s", "timeratio", "compratio");
echo strtr(sprintf(FORMAT, '', '', '', '', '', '', '', ''), ' |', '-+');
foreach($compressions as $comp) {
if(!isset($results[$comp])) {
$results[$comp] = [
'runs' => 0,
'cpu' => 0,
'size' => 0,
];
}
while($results[$comp]['runs'] < RUNS) {
passthru(sprintf('rm -Rf %s', escapeshellarg(BORG_REPO)));
passthru(sprintf(INIT_COMMAND,
escapeshellarg(BORG_BINARY),
escapeshellarg(BORG_REPO)));
$times = shell_exec('bash -c '.escapeshellarg(
sprintf('time -p '.CREATE_COMMAND,
escapeshellarg(BORG_BINARY),
escapeshellarg($comp),
escapeshellarg(BORG_REPO),
escapeshellarg(DATASET_DIR))
).' 2>&1');
preg_match_all('%^(real|user|sys) (?<time>.+)$%m', $times, $matches);
$results[$comp]['cpu'] += floatval($matches['time'][1]);
$results[$comp]['cpu'] += floatval($matches['time'][2]);
$du = shell_exec(sprintf('du -B 1 -s %s', escapeshellarg(BORG_REPO)));
preg_match('%^(?<total>[0-9]+)\s%m', $du, $match);
$results[$comp]['size'] += intval($match['total']);
++$results[$comp]['runs'];
$f = fopen(STATE_FILE, 'wb');
fwrite($f, json_encode($results));
fflush($f);
fclose($f);
}
printf(
IFORMAT,
$comp,
$results[$comp]['cpu'] / $results[$comp]['runs'],
format_bytes($results[$comp]['size'] / $results[$comp]['runs']),
format_bytes(($results['none']['size'] / $results['none']['runs']) / ($results[$comp]['cpu'] / $results[$comp]['runs'])),
($results[$comp]['cpu'] / $results[$comp]['runs']) / ($results['none']['cpu'] / $results['none']['runs']),
($results[$comp]['size'] / $results[$comp]['runs']) / ($results['none']['size'] / $results['none']['runs'])
);
}
function format_bytes(int $b, string $prec = '8.3') {
static $suffixes = [ 'B', 'K', 'M', 'G', 'T', 'P', 'E' ];
list($total, $decimals) = explode('.', $prec);
$cutoff = pow(10, $total - $decimals - 1);
$i = 0;
while($b >= $cutoff) {
$b /= 1000;
++$i;
}
return sprintf("%".$prec."f%s", $b, $suffixes[$i]);
}
function find_mountpoint($path) {
if($del = !file_exists($path)) {
touch($path);
}
$dev = shell_exec(sprintf('df %s | tail -n1 | cut -d\' \' -f1', escapeshellarg($path)));
if($del) unlink($path);
return substr($dev, 0, -1);
}
Raw
99autobench.sh
#!/bin/sh
# Run me in single-user mode:
# systemctl isolate rescue.target
for i in 1 2 3; do
rm -Rf /tmp/borg-dataset /tmp/borg-repo /tmp/borg-compressbench.state
cp -a /tmp/borg-dataset$i /tmp/borg-dataset
./borg-compressbench | tee bcb$i.txt
mv /tmp/borg-compressbench.state bcb$i.state
done
@suuuehgi
Copy link

suuuehgi commented Feb 4, 2018 •
edited

Thanks for the work! I created a plot of your comparison within my fork. What do you think about adding it here? :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment