Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Microsoft Parsers 2.2 IIS W3C Log File Format specification for GoAccess.

Example Files for Processing W3C Microsoft Log Parser files with GoAccess

  1. msplogmergexample.cmd - is an example of how to combine multiple IIS log file into a single W3C log file using Microsoft Log Parser
  2. gomspformat.sh - this file is needed to provide specifiers for GoAccess to process the W3C file generated by the Log Parser.
  3. example.sh - is an example of how to process the W3C file outputted by the log parser with GoAccess

If you create shell script yourslf ensure that the script created are not in DOS format. DOS formatted script will throw an error message with "command not found" in Cygwin. It is recommend to use Nano inside of Cygwin to make the shell script files

goaccess -f merged.log --log-format "$(cat merged.log | ./gomspformat.sh)" --date-format '%Y-%m-%d' --time-format '%H:%M:%S'
#!/usr/bin/env sh
while read line; do
if [[ $line == \#Fields:* ]]; then
line=${line/\#Fields: /}
line=${line/LogFilename/%^}
line=${line/LogRow/%^}
line=${line/date/%d}
line=${line/time/%t}
line=${line/c-ip/%h}
line=${line/cs-username/%^}
line=${line/s-sitename/%^}
line=${line/s-computername/%^}
line=${line/s-ip/%^}
line=${line/s-port/%^}
line=${line/cs-method/%m}
line=${line/cs-uri-stem/%U}
line=${line/cs-uri-query/%^}
line=${line/sc-status/%s}
line=${line/sc-substatus/%^}
line=${line/sc-win32-status/%^}
line=${line/sc-bytes/%b}
line=${line/cs-bytes/%^}
line=${line/time-taken/%L}
line=${line/cs-version/%^}
line=${line/cs-host/%^}
line=${line/cs(User-Agent)/%u}
line=${line/cs(Cookie)/%^}
line=${line/cs(Referer)/%R}
line=${line/s-event/%^}
line=${line/s-process-type/%^}
line=${line/s-user-time/%^}
line=${line/s-kernel-time/%^}
line=${line/s-page-faults/%^}
line=${line/s-total-procs/%^}
line=${line/s-active-procs/%^}
line=${line/s-stopped-procs/%^}
echo $line
exit;
fi
done
"c:\Program Files (x86)\Log Parser 2.2\LogParser.exe" -o:w3c "select * into merged.log from u_ex*.log"
@barrett092

This comment has been minimized.

Copy link

@barrett092 barrett092 commented Jun 13, 2018

Oddly I did all of these steps and it begins to parse the combined log file but then fails half way...any ideas?

@barrett092

This comment has been minimized.

Copy link

@barrett092 barrett092 commented Jun 13, 2018

perhaps the w3c format should be different? my logs are software: microsoft internet information services 7.5 version: 1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment