iptables regression patch verification

iptables-regression-verification.md

Objective

To verify whether the iptables regression patches improve performance for both direct iptables executions and for garden NetOut API calls.

iptables rule addition

4.2 kernel

time (./list-addrs 1000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m2.678s
user    0m0.072s
sys     0m0.748s

time (./list-addrs 3000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m31.723s
user    0m1.244s
sys     0m24.448s

4.4 kernel with patch

time (./list-addrs 1000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m0.960s
user    0m0.012s
sys     0m0.068s

time (./list-addrs 3000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real    0m4.357s
user    0m0.060s
sys     0m0.208s

3.13 kernel numbers

note: These numbers were collected in original regression report, on different machine with similar specs and are provided only for comparison.

time (./list-addrs 1000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real  0m0.815s
user  0m0.061s
sys   0m0.742s

time (./list-addrs 3000 | xargs -n1 iptables -A FORWARD -j ACCEPT -s)

real  0m3.975s
user  0m0.504s
sys   0m3.402s

NetOut call

This test timed:

1. Container Creation
2. Calling NetOut N times
3. Container deletion

note: This test does more than just iptables rule addition so the numbers are not directly comparable with the previous section, only against each other. The garden-runc version was built from 1.1

4.2 kernel

Ran 5 samples:
 1000 rules:
  Fastest Time: 8.784417s
  Slowest Time: 9.193594s
  Average Time: 8.921823s ± 0.143394s

Ran 5 samples:
 3000 rules:
  Fastest Time: 65.672428s
  Slowest Time: 73.553792s
  Average Time: 68.520923s ± 2.832919s

4.4 kernel with patch

Ran 5 samples:
 1000 rules:
  Fastest Time: 5.600424s
  Slowest Time: 5.866776s
  Average Time: 5.712451s ± 0.099277s

Ran 5 samples:
 3000 rules:
  Fastest Time: 19.531686s
  Slowest Time: 21.095214s
  Average Time: 20.554699s ± 0.617351s

Conclusion

There is a clear increase in speed between the 4.2 kernel which has the regression and the 4.4 patched kernel, which is closer to the 3.19 performance.

We also confirmed that garden BulkNetOut operations using iptables-restore remained fast.

list-addrs

#!/bin/bash

nRules="$1"

set -euf -o pipefail

if [ -z "$nRules" ]; then
        echo "specify an integer number of addresses to generate"
        exit 1
fi

if [ "$nRules" -eq "$nRules" 2>/dev/null ]
then
        echo -n ""
else
        echo "specify an integer number of addresses to generate"
        exit 1
fi

for i in $(seq 0 $(( $nRules - 1)) ); do
        lowbyte="$(( $i % 250 ))"
        nextbyte="$(( $i / 250 ))"
        addr="10.10.${nextbyte}.${lowbyte}"
        echo "$addr"
done