Parser for ripe atlas dns measurements

README.md

Usage

I ran this with some of the json files provided and it dumps out (in a somewhat readable/searchable text format) the measurement data to make reading it a bit easier.

You can run it like this...

$ pip install -r requirements.pip
$ python parse.py [path to json blob from ripe probes]

It generates output that looks like this. Each new measurement is printed out and the results are broken down by the query/response pairs and what was contained within them.

Measurement 12465915 from New Zealand
        Query: dnsseed.monacoin.org. IN A
                Answer:
                        dnsseed.monacoin.org. [1]=> 104.156.238.203
                        dnsseed.monacoin.org. [1]=> 202.181.101.205
                        dnsseed.monacoin.org. [1]=> 104.233.122.169
                        dnsseed.monacoin.org. [1]=> 124.39.4.147
                        dnsseed.monacoin.org. [1]=> 199.127.108.161
                        dnsseed.monacoin.org. [1]=> 49.212.166.181
                        dnsseed.monacoin.org. [1]=> 113.146.68.251
                        dnsseed.monacoin.org. [1]=> 128.199.254.216
                        dnsseed.monacoin.org. [1]=> 219.117.248.55
                        dnsseed.monacoin.org. [1]=> 144.76.3.136
                        dnsseed.monacoin.org. [1]=> 153.120.39.89
                        dnsseed.monacoin.org. [1]=> 128.199.214.168
                Authority:
                Additional:

parse.py

import json
import io
import sys
import dns.message
import base64


def main(path):
    measurements = []

    # Parse out the results from the json file
    with open(path) as fh:
        for line in fh:
            measurements.extend(json.loads(line)[0])

    for mment in measurements:
        print "Measurement %d from %s" % (mment["msm_id"], mment["country"])

        for result in mment["resultset"]:
            # Parse and print the query details
            query = dns.message.from_wire(base64.b64decode(result["qbuf"]))
            print "\tQuery: %s" % (query.question[0])

            if "result" not in result:
                continue

            # Parse and print the response
            answer = dns.message.from_wire(base64.b64decode(result["result"]["abuf"]))
            print "\t\tAnswer:"
            for rrset in answer.answer:
                for rr in rrset:
                    print "\t\t\t%s [%s]=> %s" % (rrset.name, rrset.rdtype, rr)
            print "\t\tAuthority:"
            for rrset in answer.authority:
                for rr in rrset:
                    print "\t\t\t%s [%s]=> %s" % (rrset.name, rrset.rdtype, rr)
            print "\t\tAdditional:"
            for rrset in answer.additional:
                for rr in rrset:
                    print "\t\t\t%s [%s]=> %s" % (rrset.name, rrset.rdtype, rr)

        print "---"


if __name__ == "__main__":
    main(sys.argv[1])

requirements.pip

dnspython==1.16.0