Skip to content

Instantly share code, notes, and snippets.

@vkhitev
Last active October 18, 2017 20:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vkhitev/93c1de5490e5c5b747792e64b488710a to your computer and use it in GitHub Desktop.
Save vkhitev/93c1de5490e5c5b747792e64b488710a to your computer and use it in GitHub Desktop.
Dremio & Spark & Kotlin - output
root
|-- host: string (nullable = true)
|-- time: timestamp (nullable = true)
|-- verb: string (nullable = true)
|-- resource: string (nullable = true)
|-- status: short (nullable = true)
|-- bytes: long (nullable = true)
Statistics based on the content size
+-------+------------------+
|summary| bytes|
+-------+------------------+
| count| 284342|
| mean|21291.314814554305|
| stddev| 79262.93211579716|
| min| 0|
| max| 3421948|
+-------+------------------+
Count of requests grouped by status code
+------+------+
|status| count|
+------+------+
| 200|257861|
| 302| 7343|
| 304| 17476|
| 403| 9|
| 404| 1653|
+------+------+
Hosts that accesed the server more than 700 times
+--------------------+-----+
| host|count|
+--------------------+-----+
|piweba3y.prodigy.com| 2330|
| alyssa.prodigy.com| 1362|
|piweba1y.prodigy.com| 1076|
|disarray.demon.co.uk| 1028|
|piweba4y.prodigy.com| 946|
| edams.ksc.nasa.gov| 928|
|www-d4.proxy.aol.com| 867|
|www-d3.proxy.aol.com| 823|
|www-b6.proxy.aol.com| 781|
|www-b5.proxy.aol.com| 776|
| poppy.hensa.ac.uk| 764|
| gn2.getnet.com| 757|
|www-a2.proxy.aol.com| 737|
+--------------------+-----+
Top 10 error paths
+-------------------------------------------------+-----+
|resource |count|
+-------------------------------------------------+-----+
|/pub/winvn/readme.txt |144 |
|/pub/winvn/release.txt |121 |
|/images/Nasa-logo.gif |85 |
|/shuttle/resources/orbiters/atlantis.gif |58 |
|/elv/DELTA/uncons.htm |56 |
|/shuttle/missions/sts-68/ksc-upclose.gif |51 |
|/shuttle/missions/STS-69/mission-STS-69.html |45 |
|/://spacelink.msfc.nasa.gov |36 |
|/history/apollo/sa-1/sa-1-patch-small.gif |36 |
|/shuttle/missions/sts-71/images/KSC-95EC-0916.txt|31 |
+-------------------------------------------------+-----+
only showing top 10 rows
Number of unique hosts: 19079
Unique daily hosts
+-----+---+-----+
|month|day|count|
+-----+---+-----+
| 7| 1| 5192|
| 7| 2| 4859|
| 8| 4| 4191|
| 8| 3| 3222|
| 8| 1| 2582|
| 8| 5| 1406|
| 7| 3| 634|
+-----+---+-----+
Average number of daily requests per host
+-----+---+------------------+
|month|day| avg|
+-----+---+------------------+
| 8| 4| 14.20734908136483|
| 8| 1| 13.16576297443842|
| 8| 3|12.838919925512105|
| 8| 5|12.495732574679943|
| 7| 1|12.461286594761171|
| 7| 2|12.399053303148795|
| 7| 3|10.919558359621451|
+-----+---+------------------+
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment