iAnatoly

Motivation

At some point, you might find yourself in need to transfer a file (i.e. a dump of the database, or a backup tarball) quickly over a WAN link. Unfortunately, regular copy speeds are impaired by TCP protocol and latency and speed of light - and link bandwidth has very limited effect on the transfer speeds. However, there is a workaround: one can use UDP-based file transfer (UDT). Here is a proof-of-concept experiment that demonstrates the transfer speed improvement.

Experiment

Create a large file to transfer:

iAnatoly

cat ~/.kube/config | grep client-certificate | sed -e ‘s/    client-certificate-data: //’ | base64 -d | openssl x509 -in - -noout -text

iAnatoly

# see https://github.com/iovisor/bpftrace/blob/master/docs/tutorial_one_liners.md
# histogram for udp_recvmsg timing
sudo bpftrace -e 'kprobe:udp_recvmsg { @start[tid] = nsecs; } kretprobe:udp_recvmsg /@start[tid]/ { @ns[comm] = hist(nsecs - @start[tid]); delete(@start[tid]); }'
# histogram for retval of udp_recvmsg (which is message size in bytes)
sudo bpftrace -e 'kretprobe:udp_recvmsg { @bytes[comm] = hist(retval); }'

iAnatoly

#!/usr/bin/env python3
#
# Ad-hoc monitoring for time_squeeze changes in /proc/net/softnet_stat .
#
# usage: ./softnetstat.py <refresh_interval_seconds>
#
# see https://blog.packagecloud.io/eng/2016/06/22/monitoring-tuning-linux-networking-stack-receiving-data/
# 
from time import sleep
from datetime import datetime

iAnatoly

for proc in /proc/*; do cat $proc/smaps 2>/dev/null | awk -v proc="${proc}" '/Swap/{swap+=$2/1024}END{if (swap>1) { "readlink "proc"/exe"|getline c; print int(swap+0.5), "\tMB\t", c }}'; done | sort -n

iAnatoly

bpftrace -e 'kprobe:swap_read*,kprobe:swap_write* {@[comm, pid] = count();} interval:s:5{ time(); print(@); clear(@); }'

iAnatoly

> CREATE RETENTION POLICY two_days on weather duration 2d replication 1 default
> CREATE RETENTION POLICY year on weather duration 52w replication 1
> create continuous query cq_30m on weather BEGIN SELECT mean(*) into year.:MEASUREMENT from two_days./.*/ group by time(30m),* EN

iAnatoly

SELECT temp *9/5+32 
FROM 
    (SELECT median("temperature") AS "temp" FROM "weather" WHERE $timeFilter GROUP BY time(5m)),
    (SELECT median("mean_temperature") AS "temp" FROM "year"."weather" WHERE $timeFilter GROUP BY time(1h))
GROUP BY "source" fill(null)

iAnatoly

 cp /etc/wpa_supplicant/ifupdown.sh /etc/ifplugd/action.d/ifupdown

iAnatoly

# cat > /etc/systemd/wifi-resume.service
[Unit]
Description=Restart networkmanager at resume
After=suspend.target
After=hibernate.target
After=hybrid-sleep.target

[Service]
Type=oneshot
ExecStart=/bin/systemctl restart network-manager.service