This blog benefits a lot from the helpful guidance from mdda and wangruohui.
The strategy presented below applies to the default desktop environment GNOME3 of Fedora 27, but may not work for others such as KDE Plasma. For instance, KDE tends to crash after installation of NVIDIA drivers. Actually, this is also confirmed by my own recent experience with Fedora KDE Plasma.
| #!/usr/bin/env python | |
| import gzip | |
| import os | |
| import sys | |
| import re | |
| INPUT_DIR = "nginx-logs" | |
| lineformat = re.compile(r"""(?P<ipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) - - \[(?P<dateandtime>\d{2}\/[a-z]{3}\/\d{4}:\d{2}:\d{2}:\d{2} (\+|\-)\d{4})\] ((\"(GET|POST) )(?P<url>.+)(http\/1\.1")) (?P<statuscode>\d{3}) (?P<bytessent>\d+) (["](?P<refferer>(\-)|(.+))["]) (["](?P<useragent>.+)["])""", re.IGNORECASE) |
| # -*- coding: utf-8 -*- | |
| """From a Query.all(), turn this result to a pandas DataFrame | |
| Table creation and example data come from the official SQLAlchemy ORM | |
| tutorial at http://docs.sqlalchemy.org/en/latest/orm/tutorial.html | |
| Just take a look at the 'query_to_dict' function and the last part of the __main__. | |
| """ |
