Scripts to generate download speed test graphs

README.me

# Scripts to generate pretty graphs

### Outline

Use create.sh to initialise an empty database
Add test.sh to a cron job. It runs the tests at the time you specify. My crontab looks like

    17 0,4,8,12,16,20 * * * /path/to/test.sh
  
This runs every 4 hours. There are possibly better ways to do this.

test.sh will call run_test.sh to run the actual tests, and pipe the output to a log file. It will then call mk_rrd.py with that log file to update the rrd database.

Lastly mk_image.sh will create an image. I run this manually about once a day, and update the image on the server.

Oh, and speedtest_cli.py is from https://github.com/sivel/speedtest-cli but I tweaked it to give the hostname of the server I'm hitting, so I can traceroute it. You can get my modified speedtest_cli.py from https://radelaide.strudel-hound.com/speedtest_cli.py because apparently copy+pasting it into a gist completely messes up the formatting. I hate auto-indenters that can't be disabled.

create.sh

#!/usr/bin/env bash
rrdtool create db.rrd \
  --start 1489766400 \
  --step 14400 \
  DS:a_d:GAUGE:28800:0:25 \  # I'm on a 25/5 plan, so I chose to cap things at these speeds.
  DS:a_u:GAUGE:28800:0:5 \
  DS:c_d:GAUGE:28800:0:25 \
  DS:c_u:GAUGE:28800:0:5 \
  DS:s_d:GAUGE:28800:0:25 \
  DS:s_u:GAUGE:28800:0:5 \
  RRA:MAX:0.5:1:2500

mk_rrd.py

#!/usr/bin/env python2

import sys
import datetime
import rrdtool
DATE_FORMAT = "%Y-%m-%dT%H:%M:%S"
RRD_FILE = "/home/enigma/speedtests/db.rrd"
epoch = datetime.datetime.utcfromtimestamp(0)


def get_mbits(logfile):
  mbits = []
  with open(logfile, "r") as inp:
    for line in inp:
      if "Mbit" in line:
        mbits.append(line.split()[1])
  return map(float, mbits)
  
for logfile in sys.argv[1:]:
    datestring = logfile[:-4]
    dt = datetime.datetime.strptime(datestring[:-6], DATE_FORMAT)
    speeds = get_mbits(logfile)
    time_diff = dt - epoch
    time_diff_seconds = time_diff.total_seconds()
    data = ('%d:' % (time_diff_seconds)) + ":".join(map(str, speeds)) 
    rrdtool.update(RRD_FILE, data)

run_test.sh

#!/usr/bin/env bash
SERVERS="2169 3864 3914"
TMPFILE=$(mktemp)
echo -n "Starting tests at "
date
for SERVER in ${SERVERS} ; do
  /path/to/speedtests/speedtest_cli.py --server ${SERVER} | tee ${TMPFILE}
  IP=$(cat ${TMPFILE} | awk -F '<' '/^Hosted by / {print $2}' | awk -F '>' '{print $1}')
  traceroute ${IP}
done
rm ${TMPFILE}
ping -c 100 ftp.iinet.net.au
ping -c 100 speedtest.net
ping -c 100 speedcheck.cdn.on.net
traceroute ftp.iinet.net.au
traceroute speedtest.net
traceroute speedcheck.cdn.on.net

test.sh

#!/usr/bin/env bash
cd /path/to/speedtests/
DATETIME=$(date -Iseconds)
./run_test.sh | tee ${DATETIME}.log | egrep '(Hosted by|Mbit)'
# Note that I grep out the interesting bits above, cron will automatically mail them to me.
./mk_rrd.py ${DATETIME}.log