iptables regression patch verification

iptables-regression-verification.md

Objective

In this gist, we verified that the iptables regression patches improved performance for both direct iptables executions and for garden NetOut API calls.

In this gist, we attempt to verify that the performance improvements are still seen for the Release Candidate 4.4.0-63.84~14.04.1.

iptables rule addition

4.2 kernel

time (./list-addrs 1000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m2.678s
user    0m0.072s
sys     0m0.748s

time (./list-addrs 3000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m31.723s
user    0m1.244s
sys     0m24.448s

4.4 kernel with patch (4.4.0-59.80~14.04.1hf121102v20170109b2)

time (./list-addrs 1000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m0.960s
user    0m0.012s
sys     0m0.068s

time (./list-addrs 3000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m4.357s
user    0m0.060s
sys     0m0.208s

4.4 RC kernel with patch (4.4.0-63.84~14.04.1)

time (./list-addrs 1000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m0.865s
user    0m0.008s
sys     0m0.060s

time (./list-addrs 3000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m4.196s
user    0m0.052s
sys     0m0.176s

3.13 kernel numbers

note: These numbers were collected in original regression report, on different machine with similar specs and are provided only for comparison.

time (./list-addrs 1000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real  0m0.815s
user  0m0.061s
sys   0m0.742s

time (./list-addrs 3000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real  0m3.975s
user  0m0.504s
sys   0m3.402s

NetOut call

This test timed:

1. Container Creation
2. Calling NetOut N times
3. Container deletion

note: This test does more than just iptables rule addition so the numbers are not directly comparable with the previous section, only against each other. The garden-runc version was built from 1.1

4.2 kernel

Ran 5 samples:
 1000 rules:
  Fastest Time: 8.784417s
  Slowest Time: 9.193594s
  Average Time: 8.921823s ± 0.143394s

Ran 5 samples:
 3000 rules:
  Fastest Time: 65.672428s
  Slowest Time: 73.553792s
  Average Time: 68.520923s ± 2.832919s

4.4 kernel with patch (4.4.0-59.80~14.04.1hf121102v20170109b2)

Ran 5 samples:
 1000 rules:
  Fastest Time: 5.600424s
  Slowest Time: 5.866776s
  Average Time: 5.712451s ± 0.099277s

Ran 5 samples:
 3000 rules:
  Fastest Time: 19.531686s
  Slowest Time: 21.095214s
  Average Time: 20.554699s ± 0.617351s

4.4 RC kernel with patch (4.4.0-63.84~14.04.1)

  Ran 5 samples:
  1000 rules:
    Fastest Time: 4.968385s
    Slowest Time: 5.545675s
    Average Time: 5.247120s ± 0.200444s

  3000 rules:
    Fastest Time: 17.982021s
    Slowest Time: 19.337441s
    Average Time: 18.897682s ± 0.484862s

Conclusion

The performance improvements are still present in the 4.4.0-63.84~14.04.1 RC. We also confirmed that garden BulkNetOut operations using iptables-restore remained fast.

list-addrs

#!/bin/bash

nRules="$1"

set -euf -o pipefail

if [ -z "$nRules" ]; then
        echo "specify an integer number of addresses to generate"
        exit 1
fi

if [ "$nRules" -eq "$nRules" 2>/dev/null ]
then
        echo -n ""
else
        echo "specify an integer number of addresses to generate"
        exit 1
fi

for i in $(seq 0 $(( $nRules - 1)) ); do
        lowbyte="$(( $i % 250 ))"
        nextbyte="$(( $i / 250 ))"
        addr="10.10.${nextbyte}.${lowbyte}"
        echo "$addr"
done