lmgrd log file parser

parse_lmgrd.py

#!/usr/bin/env python
# The script to parse a lmgrd log and give some license usage metrics
# 
# Copyright (c) 2014 Grigory Rechistov <grigory.rechistov@phystech.edu>
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without 
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, 
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice, 
# this list of conditions and the following disclaimer in the documentation 
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, 
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE 
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, 
# EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

import sys
import re

def main():
    lines = []
    try:
        with open (sys.argv[1], "r") as inp:
            lines=inp.readlines()
    except IOError as e:
        print e
        print "Usage: %s logfile" % sys.argv[0]
        return 1
    vendors = set()
    hosts = set()
    features = set()
    #ins  = dict() # TODO: count IN records too.
    unsupported = set()
    
    outs = dict()
    for l in lines:
        # Look for lines like this:
        # 15:13:02 (simics) OUT: "x86-nehalem" proskin@lucy  (8 licenses) 
        # 15:20:27 (simics) IN: "simics" proskin@lucy  
        #print ">>> %s" % l
        m = re.search(r'\((\w+)\)\s+OUT:\s+"(.+)"\s+([^\s]+@[^\s]+)', l)
        if m:
            vendor  = m.group(1)
            feature = m.group(2)
            user    = m.group(3).split("@")[0]
            host    = m.group(3).split("@")[1]
            # Now try to get how many licenses has been acquired
            m = re.search(r'\((\d+)\s+licenses\)', l)
            if m: n_lic = int(m.group(1))
            else: n_lic = 1
            #print ">>> OUT vendor = %s, feature = %s, n_lic = %d, user = %s, host = %s" %\
                #(vendor, feature, n_lic, user, host)
            vendors.add(vendor)
            hosts.add(host)
            features.add(feature)
            if not (user in outs):
                outs[user] = dict()
            if not (feature in outs[user]):
                outs[user][feature] = n_lic
            else:
                outs[user][feature] = outs[user][feature] + n_lic
        # Look for requests of UNSUPPORTED features.
        m = re.search(r'\((\w+)\)\s+UNSUPPORTED:\s+"(.+)"\s+.*([^\s]+@[^\s]+)', l)
        if m:
            vendor  = m.group(1)
            feature = m.group(2)
            user    = m.group(3).split("@")[0]
            host    = m.group(3).split("@")[1]
            print ">>> UNSUPPORTED vendor = %s, feature = %s, n_lic = %d, user = %s, host = %s" %\
                (vendor, feature, n_lic, user, host)
            unsupported.add(feature)

    # Dump stats
    print "All vendors:  ", [l for l in vendors]
    print "All hosts:    ", [l for l in hosts]
    print "All features: ", [l for l in features]
    print "All users:    ", outs.keys()
    
    # Users and features they've checked out, in total
    for u in outs.keys():
        print "User %s total checkouts:" % u
        for f in outs[u].keys():
            print "{:<32} {:>5}".format(f, outs[u][f])
        print ""

    if len(unsupported) != 0:
        print "Requested unsupported features:", [l for l in unsupported]
    return 0
        
if __name__ == "__main__":
    main()

Hi
Can you please help me to understand How to use this?
I am new to this.

Hi Can you please help me to understand How to use this? I am new to this.

Run ./parse_lmgrd.py <your log file> and receive statistics on license usage as output.

The script was written for Python 2 however, which is no longer supported. You will have to update it to run correctly with modern Python 3.

Hi Grigory Rechistov,
I got a message under parse_lmgrd.py running.
Could you please check this error?
Thank you.

version: Python 2.6.6
Error message:
Traceback (most recent call last):
File "./parse_lmgrd.py", line 103, in
main()
File "./parse_lmgrd.py", line 95, in main
print "{:<32} {:>5}".format(f, outs[u][f])
ValueError: zero length field name in format

@hyunseokjo00 I do not have Python 2.6.6 available anywhere, so I cannot test it.

Reading this, it comes to my mind that explicit addition of indices into format specifiers should help.

Thy this for line 95:

print "{0:<32} {1:>5}".format(f, outs[u][f])

Alternatively, upgrade to Python 2.7 which should support the used format syntax.

Hi Grigory Rechistov,
I'm working on the Linux machine(Red Hat Enterprise Linux Server release 6.9).
Your new code works fine. : print "{0:<32} {1:>5}".format(f, outs[u][f])
Thank you so much.
Thanks

Hi Grigory Rechistov,
I got the following result.
I have an additional question. I wonder if 59 and 5 mean hours.
Thanks,

User hyunseokjo00 total checkouts:
feature1 59
feature2 5

I wonder if 59 and 5 mean hours.

This current form of the script counts only the number of "OUT" events. To deduce how long a feature has been checked out, the logic has to be changed to: 1) parse the timestamps; 2) match the "IN" and "OUT" events and deduce the time periods between them.