Skip to content

Instantly share code, notes, and snippets.

@jmeggitt

jmeggitt/documentation_issues.md Secret

Created December 5, 2022 04:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jmeggitt/08375285b40e3393da49a261b6b65b52 to your computer and use it in GitHub Desktop.
Save jmeggitt/08375285b40e3393da49a261b6b65b52 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
documentation_issues.md

Documentation Inconsistencies

NTP (142426 samples)

  • dst_port is not always present
    • Possibly related to using IPv6? (unchecked)
    • Example: {"fw":5080,"mver":"2.6.2","lts":58,"dst_name":"2001:67c:2e8:14:ffff::229","dst_addr":"2001:67c:2e8:14:ffff::229","src_addr":"2a00:6020:4808:2300::97c","proto":"UDP","af":6,"li":"no","version":4,"mode":"server","stratum":1,"poll":8,"precision":0.0000038147,"root-delay":0,"root-dispersion":0.00108337,"ref-id":"GPS","ref-ts":3878842875.0368924141,"result":[{"origin-ts":3878842881.1663661003,"receive-ts":3878842881.1689414978,"transmit-ts":3878842881.1691637039,"final-ts":3878842881.1769552231,"rtt":0.010367,"offset":0.002608},{"origin-ts":3878842881.1770367622,"receive-ts":3878842881.1798114777,"transmit-ts":3878842881.1800312996,"final-ts":3878842881.1877942085,"rtt":0.010537,"offset":0.002494},{"origin-ts":3878842881.1878881454,"receive-ts":3878842881.1901931763,"transmit-ts":3878842881.1904497147,"final-ts":3878842881.1982898712,"rtt":0.010145,"offset":0.002768}],"msm_id":2048610,"prb_id":1004352,"timestamp":1669854081,"msm_name":"Ntp","from":"2a00:6020:4808:2300::97c","type":"ntp","group_id":2048610}
  • ttr is not always present
    • dst_name is already an IP, so it may not have gone through domain name resolution
    • Example: {"fw":5080,"mver":"2.6.2","lts":58,"dst_name":"2001:67c:2e8:14:ffff::229","dst_addr":"2001:67c:2e8:14:ffff::229","src_addr":"2a00:6020:4808:2300::97c","proto":"UDP","af":6,"li":"no","version":4,"mode":"server","stratum":1,"poll":8,"precision":0.0000038147,"root-delay":0,"root-dispersion":0.00108337,"ref-id":"GPS","ref-ts":3878842875.0368924141,"result":[{"origin-ts":3878842881.1663661003,"receive-ts":3878842881.1689414978,"transmit-ts":3878842881.1691637039,"final-ts":3878842881.1769552231,"rtt":0.010367,"offset":0.002608},{"origin-ts":3878842881.1770367622,"receive-ts":3878842881.1798114777,"transmit-ts":3878842881.1800312996,"final-ts":3878842881.1877942085,"rtt":0.010537,"offset":0.002494},{"origin-ts":3878842881.1878881454,"receive-ts":3878842881.1901931763,"transmit-ts":3878842881.1904497147,"final-ts":3878842881.1982898712,"rtt":0.010145,"offset":0.002768}],"msm_id":2048610,"prb_id":1004352,"timestamp":1669854081,"msm_name":"Ntp","from":"2a00:6020:4808:2300::97c","type":"ntp","group_id":2048610}
  • Replies can sometimes contain no fields other than a timeout ({"x":"*"})
    • It seems likely this is just an undocumented state for the result
    • Example: {"fw":4790,"lts":1382,"dst_name":"pool.ntp.org","dst_addr":"193.34.155.3","src_addr":"192.168.1.4","proto":"UDP","af":4,"result":[{"x":"*"},{"x":"*"},{"x":"*"}],"msm_id":2048605,"prb_id":4133,"timestamp":1669856007,"msm_name":"Ntp","from":"134.249.176.157","type":"ntp","group_id":2048605}
  • Replies can sometimes contain no fields other than an error (Ex: {"error":"name resolution failed: non-recoverable failure in name resolution"})
    • It seems likely this is just an undocumented state for the result
    • Example: {"fw":5040,"mver":"2.4.1","lts":1309,"dst_name":"pool.ntp.org","ttr":15028.467987,"proto":"UDP","af":4,"result":[{"error":"name resolution failed: non-recoverable failure in name resolution"}],"msm_id":2048605,"prb_id":1004454,"timestamp":1669853322,"msm_name":"Ntp","from":"62.122.140.140","type":"ntp","group_id":2048605}
  • Response can sometimes contain dst_addr field
    • It seems likely this is just an undocumented value. The example value in the documentation does contain this field even though it is not listed.
    • Example: {"fw":5080,"mver":"2.6.2","lts":58,"dst_name":"2001:67c:2e8:14:ffff::229","dst_addr":"2001:67c:2e8:14:ffff::229","src_addr":"2a00:6020:4808:2300::97c","proto":"UDP","af":6,"li":"no","version":4,"mode":"server","stratum":1,"poll":8,"precision":0.0000038147,"root-delay":0,"root-dispersion":0.00108337,"ref-id":"GPS","ref-ts":3878842875.0368924141,"result":[{"origin-ts":3878842881.1663661003,"receive-ts":3878842881.1689414978,"transmit-ts":3878842881.1691637039,"final-ts":3878842881.1769552231,"rtt":0.010367,"offset":0.002608},{"origin-ts":3878842881.1770367622,"receive-ts":3878842881.1798114777,"transmit-ts":3878842881.1800312996,"final-ts":3878842881.1877942085,"rtt":0.010537,"offset":0.002494},{"origin-ts":3878842881.1878881454,"receive-ts":3878842881.1901931763,"transmit-ts":3878842881.1904497147,"final-ts":3878842881.1982898712,"rtt":0.010145,"offset":0.002768}],"msm_id":2048610,"prb_id":1004352,"timestamp":1669854081,"msm_name":"Ntp","from":"2a00:6020:4808:2300::97c","type":"ntp","group_id":2048610}
  • root-dispersion sometimes appears in replies
    • Example: {"fw":5080,"mver":"2.6.2","lts":62,"dst_name":"78.219.181.254","dst_addr":"78.219.181.254","src_addr":"139.165.223.130","proto":"UDP","af":4,"li":"no","version":4,"mode":"server","stratum":3,"poll":8,"precision":0.0000019074,"root-delay":0.0149689,"root-dispersion":0.0735168,"ref-id":"ac1202e3","ref-ts":3878840001.4968299866,"result":[{"origin-ts":3878841621.0064930916,"receive-ts":3878841620.4742107391,"transmit-ts":3878841620.4744205475,"final-ts":3878841621.0429911613,"rtt":0.036289,"offset":0.550426},{"root-dispersion":0.0735321,"origin-ts":3878841621.0434770584,"receive-ts":3878841620.5098366737,"transmit-ts":3878841620.5100636482,"final-ts":3878841621.0785179138,"rtt":0.034814,"offset":0.551047},{"root-dispersion":0.0735321,"origin-ts":3878841621.0790438652,"receive-ts":3878841620.5453066826,"transmit-ts":3878841620.5455169678,"final-ts":3878841621.1139855385,"rtt":0.034732,"offset":0.551103}],"msm_id":47086584,"prb_id":20003,"timestamp":1669852820,"msm_name":"Ntp","from":"139.165.223.130","type":"ntp","group_id":47086584}
  • stratum can sometimes hold string value "invalid"
    • Example: {"fw":5080,"mver":"2.6.2","lts":5571596,"dst_name":"2.pool.ntp.org","ttr":0.696362,"dst_addr":"2a01:4f8:222:2213:10::2","src_addr":"2a0e:46c4:2400:acab::2","proto":"UDP","af":6,"li":"no","version":4,"mode":"server","stratum":2,"poll":8,"precision":0.0000000596,"root-delay":0.0114288,"root-dispersion":0.0287476,"ref-id":"ed11cc5f","ref-ts":3878840860.6071987152,"result":[{"origin-ts":3878841977.7826018333,"receive-ts":3878841730.5826835632,"transmit-ts":3878841730.5827188492,"final-ts":3878841977.7908492088,"rtt":0.008211,"offset":247.204024},{"li":"unknown","stratum":"invalid","precision":1,"root-delay":0,"root-dispersion":0,"ref-id":"RATE","ref-ts":0.0,"origin-ts":3878841977.7909011841,"receive-ts":3878841977.7909011841,"transmit-ts":3878841977.7909011841,"final-ts":3878841977.7992367744,"rtt":0.008336,"offset":0.004168},{"x":"*"}],"msm_id":2048609,"prb_id":1003165,"timestamp":1669853177,"msm_name":"Ntp","from":"2a0e:46c4:2400:acab::2","type":"ntp","group_id":2048609}
  • li, mode, poll, precision, ref-id, ref-ts, root-delay, root-dispersion, version, stratum are not always present
    • Occurred when all results were timeouts. Would not have been given a value to fill this field
    • Example: {"fw":4790,"lts":1382,"dst_name":"pool.ntp.org","dst_addr":"193.34.155.3","src_addr":"192.168.1.4","proto":"UDP","af":4,"result":[{"x":"*"},{"x":"*"},{"x":"*"}],"msm_id":2048605,"prb_id":4133,"timestamp":1669856007,"msm_name":"Ntp","from":"134.249.176.157","type":"ntp","group_id":2048605}
  • li, precision, ref-id, ref-ts, root-delay, root-dispersion, stratum sometimes appear in replies
    • Example: {"fw":5080,"mver":"2.6.2","lts":5571596,"dst_name":"2.pool.ntp.org","ttr":0.696362,"dst_addr":"2a01:4f8:222:2213:10::2","src_addr":"2a0e:46c4:2400:acab::2","proto":"UDP","af":6,"li":"no","version":4,"mode":"server","stratum":2,"poll":8,"precision":0.0000000596,"root-delay":0.0114288,"root-dispersion":0.0287476,"ref-id":"ed11cc5f","ref-ts":3878840860.6071987152,"result":[{"origin-ts":3878841977.7826018333,"receive-ts":3878841730.5826835632,"transmit-ts":3878841730.5827188492,"final-ts":3878841977.7908492088,"rtt":0.008211,"offset":247.204024},{"li":"unknown","stratum":"invalid","precision":1,"root-delay":0,"root-dispersion":0,"ref-id":"RATE","ref-ts":0.0,"origin-ts":3878841977.7909011841,"receive-ts":3878841977.7909011841,"transmit-ts":3878841977.7909011841,"final-ts":3878841977.7992367744,"rtt":0.008336,"offset":0.004168},{"x":"*"}],"msm_id":2048609,"prb_id":1003165,"timestamp":1669853177,"msm_name":"Ntp","from":"2a0e:46c4:2400:acab::2","type":"ntp","group_id":2048609}

TLS (137444 samples)

  • Undocumented field err sometimes appears in response
    • Example: {"fw":5080,"mver":"2.6.2","lts":37,"dst_name":"212.72.229.170","dst_port":"443","dst_addr":"212.72.229.170","af":4,"err":"connect: No route to host","msm_id":26427708,"prb_id":28447,"timestamp":1669852801,"msm_name":"SSLCert","from":"77.250.238.13","type":"sslcert","group_id":26427708}
  • Undocumented field error sometimes appears in response
    • Example: {"fw":5080,"mver":"2.6.2","lts":24,"dst_name":"abs.twimg.com","dst_port":"443","ttr":14.41075,"dst_addr":"10.10.34.36","af":4,"error":"address not allowed","msm_id":29280691,"prb_id":1003958,"timestamp":1669852801,"msm_name":"SSLCert","from":"185.137.109.230","type":"sslcert","group_id":29280691}
  • Undocumented field dnserr sometimes appears in response
    • Example: {"fw":5080,"mver":"2.6.2","lts":2,"dst_name":"pbs.twimg.com","dst_port":"443","ttr":15003.534632,"dnserr":"non-recoverable failure in name resolution","msm_id":29280675,"prb_id":53829,"timestamp":1669852802,"msm_name":"SSLCert","from":"178.74.196.31","type":"sslcert","group_id":29280675}
  • method, ver is not always present
    • Likely related to err field being present
    • Example: {"fw":5080,"mver":"2.6.2","lts":37,"dst_name":"212.72.229.170","dst_port":"443","dst_addr":"212.72.229.170","af":4,"err":"connect: No route to host","msm_id":26427708,"prb_id":28447,"timestamp":1669852801,"msm_name":"SSLCert","from":"77.250.238.13","type":"sslcert","group_id":26427708}
  • af is not always present
    • Likely related to dnserr field being present
    • Example: {"fw":5080,"mver":"2.6.2","lts":2,"dst_name":"pbs.twimg.com","dst_port":"443","ttr":15003.534632,"dnserr":"non-recoverable failure in name resolution","msm_id":29280675,"prb_id":53829,"timestamp":1669852802,"msm_name":"SSLCert","from":"178.74.196.31","type":"sslcert","group_id":29280675}

HTTP (2992939 samples)

  • bsize, hsize, res, rt, src_addr, ver not always present in result
    • Likely related to err field being present
    • {"fw":5080,"mver":"2.6.2","lts":36749771,"result":[{"err":"connect: Connection refused","method":"GET","af":6,"dst_addr":"2606:d680:a71a:1::2"}],"uri":"http://2606:d680:a71a:1::2/4096","msm_id":38816152,"prb_id":1002927,"timestamp":1669853190,"msm_name":"HTTPGet","from":"2605:c640:911:124::3","type":"http","group_id":38816149}
  • Undocumented field ttr sometimes appears in response
    • Example: {"fw":5080,"mver":"2.6.2","lts":325,"ttr":54.650289,"result":[{"method":"GET","af":6,"dst_addr":"2606:4700::6812:142c","src_addr":"2001:1398:1:0:200:1:122:55","rt":32.216299,"res":301,"ver":"1.1","hsize":556,"bsize":0}],"uri":"http://www.ripe.net/favicon.ico","msm_id":13023,"prb_id":6554,"timestamp":1669852836,"msm_name":"HTTPGet","from":"2001:1398:1:0:200:1:122:55","type":"http"}
  • Replies can sometimes hold no other fields than dnserr ({"dnserr":"non-recoverable failure in name resolution (1)"})
    • Example: {"fw":4790,"lts":9,"result":[{"dnserr":"non-recoverable failure in name resolution (1)"}],"uri":"http://ipv6.msftncsi.com/ncsi.txt","msm_id":1003937,"prb_id":1326,"timestamp":1669855106,"msm_name":"HTTPGet","from":"2003:e3:d721:cd00:220:4aff:fec8:29f7","type":"http","group_id":1003937}

DNS (8289875 samples)

  • proto is not always present
    • Example: {"fw":5040,"mver":"2.4.1","lts":23,"resultset":[{"time":1669853988,"lts":23,"subid":1,"submax":1,"dst_addr":"127.0.0.53","dst_port":"53","af":4,"src_addr":"127.0.0.1","proto":"UDP","result":{"rt":773.707,"size":62,"abuf":"joaBgAABAAEAAAAABnNlY3VyZQZkM2E4bjMKcm9vdGNhbmFyeQNuZXQAAAEAAcAMAAEAAQAAADwABJFhFBQ=","ID":36486,"ANCOUNT":1,"QDCOUNT":1,"NSCOUNT":0,"ARCOUNT":0}}],"msm_id":8926887,"prb_id":1003460,"timestamp":1669853988,"msm_name":"Tdig","from":"198.13.59.118","type":"dns","group_id":8926887}
  • Undocumented field qt sometimes appears in results
    • Example: {"fw":5070,"mver":"2.6.1","lts":28,"dst_addr":"199.7.83.42","dst_port":"53","af":4,"src_addr":"192.168.1.183","proto":"TCP","result":{"rt":4.672,"qt":2.123,"size":857,"abuf":"6a6EAAABAAEADQAaAAAGAAEAAAYAAQABUYAAQAFhDHJvb3Qtc2VydmVycwNuZXQABW5zdGxkDHZlcmlzaWduLWdycwNjb20AeIb+6gAABwgAAAOEAAk6gAABUYAAAAIAAQAH6QAAAsAcAAACAAEAB+kAAAQBYsAeAAACAAEAB+kAAAQBY8AeAAACAAEAB+kAAAQBZMAeAAACAAEAB+kAAAQBZcAeAAACAAEAB+kAAAQBZsAeAAACAAEAB+kAAAQBZ8AeAAACAAEAB+kAAAQBaMAeAAACAAEAB+kAAAQBacAeAAACAAEAB+kAAAQBasAeAAACAAEAB+kAAAQBa8AeAAACAAEAB+kAAAQBbMAeAAACAAEAB+kAAAQBbcAewBwAAQABAAfpAAAExikABMB0AAEAAQAH6QAABMcJDsnAgwABAAEAB+kAAATAIQQMwJIAAQABAAfpAAAExwdbDcChAAEAAQAH6QAABMDL5grAsAABAAEAB+kAAATABQXxwL8AAQABAAfpAAAEwHAkBMDOAAEAAQAH6QAABMZhvjXA3QABAAEAB+kAAATAJJQRwOwAAQABAAfpAAAEwDqAHsD7AAEAAQAH6QAABMEADoHBCgABAAEAB+kAAATHB1MqwRkAAQABAAfpAAAEygwbIcAcABwAAQAH6QAAECABBQO6PgAAAAAAAAACADDAdAAcAAEAB+kAABAgAQUAAgAAAAAAAAAAAAALwIMAHAABAAfpAAAQIAEFAAACAAAAAAAAAAAADMCSABwAAQAH6QAAECABBQAALQAAAAAAAAAAAA3AoQAcAAEAB+kAABAgAQUAAKgAAAAAAAAAAAAOwLAAHAABAAfpAAAQIAEFAAAvAAAAAAAAAAAAD8C/ABwAAQAH6QAAECABBQAAEgAAAAAAAAAADQ3AzgAcAAEAB+kAABAgAQUAAAEAAAAAAAAAAABTwN0AHAABAAfpAAAQIAEH/gAAAAAAAAAAAAAAU8DsABwAAQAH6QAAECABBQMMJwAAAAAAAAACADDA+wAcAAEAB+kAABAgAQf9AAAAAAAAAAAAAAABwQoAHAABAAfpAAAQIAEFAACfAAAAAAAAAAAAQsEZABwAAQAH6QAAECABDcMAAAAAAAAAAAAAADU=","ID":59822,"ANCOUNT":1,"QDCOUNT":1,"NSCOUNT":13,"ARCOUNT":26,"answers":[{"TYPE":"SOA","NAME":".","TTL":86400,"MNAME":"a.root-servers.net.","RNAME":"nstld.verisign-grs.com.","SERIAL":2022113002}]},"msm_id":10108,"prb_id":18312,"timestamp":1669853984,"msm_name":"Tdig","from":"195.67.195.86","type":"dns"}
  • Undocumented field name sometimes appears in results
    • Example: {"fw":5070,"mver":"2.6.1","lts":63,"name":"dns5.telia.com","dst_addr":"81.228.11.68","dst_port":"53","af":4,"src_addr":"192.168.179.20","proto":"UDP","result":{"rt":41.049,"size":216,"abuf":"5x+EAAABAAEABgAADmZvcnN2YXJzbWFrdGVuAnNlAAAGAAHADAAGAAEAACowACgCbnMDbWlsAnNlAAZub2MtZm3AMniG0WcAAA4QAAABaAASdQAAACowwAwAAgABAAAHCAACwC/ADAACAAEAAAcIAAYDbnMywDLADAACAAEAAAcIAAYDbnMzwDLADAACAAEAAAcIABAEZG5zNQV0ZWxpYQNjb20AwAwAAgABAAAHCAAHBGRuczbAmsAMAAIAAQAABwgAFAVwaXRlYQNkbnMEc3dpcANuZXQA","ID":59167,"ANCOUNT":1,"QDCOUNT":1,"NSCOUNT":6,"ARCOUNT":0,"answers":[{"TYPE":"SOA","NAME":"forsvarsmakten.se.","TTL":10800,"MNAME":"ns.mil.se.","RNAME":"noc-fm.mil.se.","SERIAL":2022101351}]},"msm_id":9318258,"prb_id":26955,"timestamp":1669853909,"msm_name":"Tdig","from":"46.90.146.113","type":"dns","group_id":9318258}

PING (30041836 samples)

  • ttr not always present
    • Example: {"fw":5040,"mver":"2.4.0","lts":55439788,"dst_name":"193.0.14.129","af":4,"dst_addr":"193.0.14.129","src_addr":"2.78.40.133","proto":"ICMP","ttl":56,"size":32,"result":[{"rtt":40.607546},{"rtt":40.379923},{"rtt":40.398052}],"dup":0,"rcvd":3,"sent":3,"min":40.379923,"max":40.607546,"avg":40.4618403333,"msm_id":1001,"prb_id":1001967,"timestamp":1669852812,"msm_name":"Ping","from":"2.78.40.133","type":"ping","step":240}
  • Undocumented field step sometimes (always?) appears in response
    • Example: {"fw":5040,"mver":"2.4.0","lts":55439788,"dst_name":"193.0.14.129","af":4,"dst_addr":"193.0.14.129","src_addr":"2.78.40.133","proto":"ICMP","ttl":56,"size":32,"result":[{"rtt":40.607546},{"rtt":40.379923},{"rtt":40.398052}],"dup":0,"rcvd":3,"sent":3,"min":40.379923,"max":40.607546,"avg":40.4618403333,"msm_id":1001,"prb_id":1001967,"timestamp":1669852812,"msm_name":"Ping","from":"2.78.40.133","type":"ping","step":240}
    • Example: {"fw":5080,"mver":"2.6.2","lts":16786813,"dst_name":"ch-us-west-1.s3.amazonaws.com","ttr":3.467857,"af":4,"dst_addr":"52.219.113.58","src_addr":"72.18.149.125","proto":"ICMP","ttl":235,"size":48,"result":[{"rtt":28.005821},{"rtt":28.065709},{"rtt":28.10466}],"dup":0,"rcvd":3,"sent":3,"min":28.005821,"max":28.10466,"avg":28.05873,"msm_id":47270037,"prb_id":1002511,"timestamp":1669853093,"msm_name":"Ping","from":"72.18.149.125","type":"ping","group_id":47270037,"step":null}
  • srcaddr field sometimes appears in replies
    • Example: {"fw":4600,"lts":2,"dst_name":"2600:3c01::f03c:91ff:fec8:52fe","dst_addr":"2600:3c01::f03c:91ff:fec8:52fe","af":6,"proto":"ICMP","size":12,"result":[{"x":"*","srcaddr":"2a02:4260:1004::4"},{"x":"*"},{"x":"*"}],"dup":0,"rcvd":0,"sent":3,"min":-1,"max":-1,"avg":-1,"msm_id":2028,"prb_id":17480,"timestamp":1669854388,"msm_name":"Ping","from":"2a02:4260:1004::4","type":"ping","step":240}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment